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ABSTRACT 



The National Educational Research Policy and Priorities 
Board has developed standards for the conduct and evaluation of research, 
development, and dissemination carried out under the auspices of the Office 
of Educational Research and Improvement (OERI) of the U.S. Department of 
Education. This handbook explains the scope and significance of the 
standards, which require OERI to subject applications for funding to peer 
review, to evaluate all existing projects to see how well they are 
performing, and to select promising and exemplary programs to share with 
educators across the United States , The three appendixes reproduce the full 
text of the standards as they were published in three phases (1995, 1997, and 
1998) in the "Federal Register." (SLD) 
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Foreword 



One of the chief responsibilities the Congress gave the National Educational 
Research Policy and Priorities Board (NERPPB) in its authorizing legislation was 
to review and approve standards for the conduct and evaluation of all research, 
development, and dissemination carried out under the auspices of the Office of 
Educational Research and Improvement (OERI) of the U.S. Department of Educa- 
tion. These standards were designed to assure that such activities meet the highest 
criterion of professional excellence. 

The development of the standards was a lengthy process. The Assistant Secretary 
for the Office of Educational Research and Improvement examined the procedures 
utilized by the National Institutes of Health, the National Science Foundation, and 
other federal agencies, and solicited recommendations fi'om research organiza- 
tions and fi'om the public. The standards were published in three phases over 
several years, as the law required. 

The Board has chosen to complete its mandate with this brief handbook which 
explains the scope and significance of the standards for the research community 
and the public, and reproduces in the appendices the full text of the standards as 
published in the Federal Register. We have firm expectations that the standards 
will be another powerful engine for eliciting the knowledge upon which educa- 
tional reform may continue to build. 

Kenji Hakuta 
Chair 

National Educational Research Policy and Priorities Board 
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Attaining Excellence 

Before trying a new medicine, most Americans want assurance that it has met 
certain safety and efficacy standards and earned the approval of the U.S. Food and 
Drug Administration. But what assurances are there about the reliability of educa- 
tion programs intended to improve the learning and achievement of the nation’s 
children? How can consumers judge their quality, and who will vouch for their 
effectiveness? 

If the program in question is an activity funded by the U.S. Department of 
Education’s Office of Educational Research and Improvement (OERI), educators 
and taxpayers can be assured that it has been subjected to a vigorous peer review 
process and that it meets the highest standards of professional excellence. 

OERI is the branch of the U.S. Department of Education primarily responsible for 
funding research and demonstration projects, collecting education statistics, and 
sharing information about education programs that work. 

OERI has always used a system of peer review, the use of recognized experts to 
evaluate proposed and funded research and program activities. Now, however, 
OERI has for the first time defined and published the professional qualifications 
individuals must possess in order to become peer reviewers and a consistent set of 
criteria reviewers must use to determine the success or potential for success of any 
program funded and disseminated by OERI. 

The new standards govern how OERI conducts business. All OERI activities 
must now meet a specific set of published standards that reflect the highest levels 
of professional excellence. The standards not only offer OERI a yardstick by 
which to measure quality, but also ensure that everyone is measured by the same 
yardstick. The standards also reinforce OERI’s confidence that the projects it 
funds will contribute to improving the education of all students. 

The standards can help many OERI constituents. For researchers, they offer 
guidance about OERI’s goals and criteria for awarding grants and contracts. For 
teachers and administrators, they offer reassurance that the programs OERI funds 
and the information it disseminates have met certain high requirements. For 
parents, other taxpayers, policymakers, and members of Congress, they offer a 
means of holding OERI accountable for its use of federal dollars. 
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The new standards were developed by the Assistant Secretary, his staff, and by 
members of the National Educational Research Policy and Priorities Board, which 
collaborates with the Assistant Secretary in the development of research priorities. 
The 15-member Board includes educational researchers, practitioners, parents, 
and other individuals who have expertise and broad perspectives in the education 
field. 

This handbook provides a general overview of the standards and highlights some 
of their key characteristics. Precise legal explanations of the standards are avail- 
able from the Federal Register, as indicated in the appendix to this document. 

When Does OERI Use Peer Review? 

The new standards require OERI to use peer review for three major program 
activities: to review applications for fimding; to evaluate all existing projects to 
determine how well they are performing; and to select promising and exemplary 
programs to share with educators across the country. 

Review of Applications 

To help ensure that OERI funds only high quality projects, all applications for 
OERI grants and cooperative agreements and any contract proposal for more than 
$100,000 must undergo a peer review. Generally, applications and proposals will 
be reviewed by three to five expert reviewers. 

Evaluation of Funded Projects 

In a major move to continuously improve the quality of funded projects, OERI 
now requires that these projects be evaluated at least twice after they are funded: 
at least once while the project is under way and again at its conclusion. The 
interim reviews will allow OERI to fine tune project activities and, if necessary, to 
correct any problems. In the past, projects were evaluated but not according to a 
published set of standards. All OERI research, development and dissemination 
activities, ranging from the smallest purchase orders and commissioned papers to 
the largest research projects and national research centers, will now be subjected 
to this peer review. 

Identifying Exemplary and Promising Programs 

Another of OERI’s functions is to examine education programs from across the 
country to determine whether they should be designated as “exemplary” or 
“promising” and disseminated to the field. The new standards require that each 
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program submitted for consideration be assessed by an expert panel of peer 
reviewers and judged by a common set of standards. The panels will designate a 
program as “exemplary” if empirical data demonstrate that it is effective in im- 
proving student achievement, or “promising” if there is sufficient evidence that 
the program shows promise for improving student achievement. Progreims may be 
submitted to OERI for review at any time by individuals and organizations, or 
they may be proposed by the Secretary of Education. 

Who May Serve as Peer Reviewers? 

It depends on what program is being reviewed. Generally, all peer reviewers must 
be leaders in the field and must have proven expertise in the subject matter they 
are reviewing. They also must meet stringent conflict-of-interest regulations. The 
Secretary of Education selects each peer reviewer and may solicit nominations for 
peer reviewers from professional associations, nationally recognized experts, and 
other sources. Some of the more specific qualifications include: 

• Individuals reviewing applications for fimding must have knowledge 
about education policy or practice, and theory or methodology in the 
subject they are reviewing. 

• Reviewers evaluating how well existing grants, cooperative agreements, 
and contracts are performing must have knowledge and expertise in the 
subject area being reviewed. To evaluate the performance of the recipients, 
their qualifications may include expert knowledge about theory, methods, 
and education policies and practices as well as practical experience related 
to the topic they are evaluating and in managing complex organizations. 

• Experts selected to help OERI designate “exemplary” or “promising” 
programs must have in-depth knowledge of the subject area or content of 
the program or group of programs they will be evaluating. To ensure that 
the peer reviewers represent both practice and research, each panel must 
include at least one current teacher, principal, or other school-based or 
community-based professional, and at least one individual experienced in 
evaluating educational programs. 

What Criteria Will Peer Reviewers Use? 

The answer depends on which of the three activities is being reviewed. While the 
criteria are different for each of the three, all were designed with one goal in 
mind: to ensure that everything OERI does is oriented towards its customers. 
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credible to researchers, and useful to practitioners. Here is a summary of the three 
sets of criteria. 

Criteria for Selecting Projects for Funding 

Grants and cooperative agreements are selected for funding according to a set of 
evaluation criteria specifically designed to identify the best. Peer reviewers 
consider one or more of the following; 

> Is the project nationally significant? 

> What is the quality of the project’s design? 

> What are the qualifications and potential contributions of the project 
personnel? 

> Is the proposed budget adequate to complete the project? 

> Is the management plan adequate? 

Answering these questions will help reviewers determine whether to recommend 
to OERI that the project be funded. For each competition, criteria are further 
defined to target the specific purposes and expected outcomes for funded activi- 
ties. 

Contract proposals will be judged by specific criteria, which may include but are 
not limited to; 

> Technical excellence; 

> Management capability; 

> Personnel qualifications; 

> Prior experience; 

> Pzist performance; and 

> Schedule compliance. 
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Criteria for Evaluating Funded Projects 

To help OERI determine how well each of its funded projects is performing, peer 
reviewers will examine the projects to see if each is meeting a standard set of four 
criteria. The same criteria will be used for both interim and final reviews. The 
criteria will be especially useful during interim reviews because they will allow 
OERI to assess a project’s progress and determine if technical assistance is neces- 
sary to ensure that the project achieves its objectives. 

Peer reviewers will thoroughly review each project and look for evidence that the 
project: 

> Is being well implemented and managed; 

> Is of high quality; 

> Is producing products and services that are useful; and 

> Will have an impact on the field of education. 

OERI will ask reviewers to consider additional criteria for specific types of 
projects. 

Criteria for Selecting Exemplary and Promising Programs 

If they are to use them with confidence, educators must have assurance that the 
programs OERI deems “exemplary” or “promising” truly meet these high stan- 
dards. OERI requires its peer review panels to use four criteria when examining 
programs that apply or are nominated for this honor. The standards help to assure 
that these programs are high quality, research-based programs that have provided 
evidence that they have improved teaching, learning, or both, or have demon- 
strated other worthy educational performance outcomes. To determine if a pro- 
gram should be recommended as “exemplary” or “promising,” peer reviewers 
must evaluate: 

> Evidence of a program’s success; 

> Quality of the program; 

> Educational significance; and 

> The program’s replicability. 

15 
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OERI and the National Educational Research Policy and Priorities Board have 
invested much thought and energy in developing these new standards. They 
sought input from practitioners, policymakers, researchers, the public, and other 
federal agencies in an effort to improve the quality of the agency’s programs and 
projects. The new standards will enhance OERI’s ability to provide the broad field 
of education and the nation access to programs and activities that meet the highest 
level of professional excellence. 

For further information, consult: 

• OERI’s Home page: mvw.ed.gov/qfjftces/OERI/ 

• Federal Register Home page: www.nara.gov/fedreg/ 
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DEPARTMENT OF EDUCATION 

34 CFR Part 700 

RIN 1850-AA51 

Standards for the Conduct and 
Evaluation of Activities Carried Out by 
the Office of Educational Research and 
Improvement (OERI) — Evaluation of 
Applications for Grants and 
Cooperative Agreements and 
Proposals for Contracts 

agency: Department of Education. 
action: Final Regulations. 

SUMMARY: The Assistant Secretary for 
Educational Research and Improvement 
establishes final regulations that set 
standards for the evaluation of 
applications for grants and cooperative 
agreements and proposals for contracts. 
The development of these standards is 
required by the Office of Educational 
Research and Improvement's 
authorizing legislation, the “Educational 
Research, Development, Dissemination, 
and Improvement Act of 1994." The 
standards ensure that these application 
and proposal evaluation activities meet 
the highest standards of professional 
excellence. 

EFFECTIVE DATE: These regulations take 
effect October 16, 1995. 

FOR FURTHER INFORMATION CONTACT: 
Edward J. Fuentes, U.S. Department of 
Education, 555 New Jersey Avenue, 

NW., Room 600, Washington, DC 
20208-5530. Telephone (202) 219-1895. 
Internet electronic mail address: stan- 
questions@inet.ed.gov. Individuals who 
use a telecommunications device for the 
deaf (TDD) may call the Federal 
Information Relay Service (FIRS) at 1- 
800-877-8339 between 8 a.m. and 8 
p.m.. Eastern time, Monday through 
Friday. 

SUPPLEMENTARY INFORMATION: On March 
31, 1994, President Clinton signed Pub. 
L. 103-227, which includes Title IX — 
the “Educational Research, 
Development, Dissemination, and 
Improvement Act of 1994” (the “Act”). 
The Act restructured the Office of 
Educational Research and Improvement 
(OERI) and endowed it with a broad 
mandate to conduct an array of research, 
development, dissemination, and 
improvement activities aimed at 
strengthening the education of all 
students. The Act also required the 
establishment of a National Educational 
Research Policy and Priorities Board 
(the “Board”) to work collaboratively 
vn\h the Assistant Secretary to identify 
priorities to guide the work of OERI. 

The legislation directed the Assistant 
Secretary to develop, in consultation 



with the Board, such standards as may 
be necessary to govern the conduct and 
evaluation of all research, development, 
and dissemination activities carried out 
by the Office to ensure that such 
activities meet the highest standards of 
professional excellence. The legislation 
required that the standards be 
developed in three phases. These 
regulations implement the first phase of 
the standards. The Assistant Secretary 
will publish at a later date additional 
proposed regulations to implement the 
remaining standards in accordance with 
the timelines established in the Act. The 
legislation requires the Board to review 
and approve the final standards. 

On June 7, 1995, the Assistant 
Secretary for Educational Research and 
Improvement published a notice of 
proposed rulemaking (NPRM) in the 
Federal Register (60 FR 30160). 

Analysis of Comments and Changes 

In response to the Assistant 
Secretary's invitation in the NPRM, five 
parties submitted comments on the 
proposed regulations. An analysis of the 
comments and of the changes in the 
regulations since publication of the 
NPRM follows. 

Issues are grouped according to 
subject with appropriate sections of the 
regulations referenced in parentheses. 
Substantive issues are discussed under 
the section of the regulations to which 
they pertain. In addition to the public 
comment, the comments of the Board's 
Committee on Standards are also 
addressed. That Committee met in 
public session on August 4, 1995, to 
provide final input for the Board and to 
act on the Board's behalf in approving 
the standards. Technical and other 
minor changes — and suggested changes 
the Secretary is not legally authorized to 
make under the applicable statutory 
authority — are not addressed. 

Qualifications of Peer Reviewers 
(§700.11) 

Comments: Two commenters believed 
that § 700.11 should require the majority 
of reviewers for a given application to 
meet the qualifications in 
§ 700.1l(a)(l)(i). These commenters 
were concerned that requiring 
individual reviewers to possess only 
one or more of the qualifications listed 
under § 700.11(a)(1) might result in few 
or no reviewers for a given application 
possessing demonstrated expertise in 
the subject of the competition 
(§ 700.1l(a)(l)(i)). One of these 
commenters also felt that each group of 
reviewers for a given application should 
include at least one reviewer with “in- 
depth knowledge of policy and practice 
in the field of education” 
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(§ 700.1l(a)(l)(ii)), and at least one 
reviewer with “in-depth knowledge of 
theoretical perspectives or 
methodological approaches relevant to 
the subject of the competition” 

(§ 700.1l(a)(l)(iii)). Another commenter 
felt that all reviewers for research 
projects should possess technical 
expertise regarding the theoretical and 
methodological aspects of the grant 
applications. 

Discussion: The Secretary believes it 
is important for all reviewers to possess 
each of the qualifications in 
§ 700.11(a)(1). The Board agreed that it 
is important for all reviewers to possess 
each of the qualifications in 
§ 700.11(a)(1) but recommended that 
§ 700.1l(a)(l)(ii) be modified to allow a 
reviewer to be deemed qualified on the 
basis of in-depth knowledge of policy 
“or” practice in the field of education 
rather than both. 

Changes: The Secretary has revised 
§ 700.11(a)(1) to require each reviewer 
to possess each of the qualifications of 
(a)(l)(i), (a)(l)(ii) and (a)(l)(iii) and 
replaced the word “and” in (a)(l)(ii) 
with the word “or”. 

Comments: One commenter expressed 
concern that the word “relevant” in 
§ 700.1 l(a)(l)(i) was inadequate for 
specifying that reviewers have direct 
expertise in the topic area of grant 
applications they review. 

Discussion: The Secretary agrees with 
this commenter. 

Changes: The Secretary replaced the 
words “relevant to the subject area” 
with the words “in the subject area” in 
§ 700.1l(a)(l)(i) and also in 
§700.1l(a)(l)(iii). 

Convening Reviewers to Discuss 
Unsolicited Applications (§ 700.21) 

Comments: One commenter suggested 
substituting the word “may” for the 
word “will” in § 700.21(c) so as not to 
require the convening of reviewers in all 
instances. The commenter believed that 
it may not be necessary to convene 
reviewers to discuss the strengths and 
weaknesses of unsolicited applications. 

Discussion: The Secretary believes 
that discussions of each application's 
strengths and weaknesses allows 
reviewers to share perspectives and 
provide a higher quality of review. 
Therefore, the Secreteury believes that 
such discussions should, in general, be 
required. However, the Secretary agrees 
that in the case of unsolicited 
applications, it may not be necesseiry to 
convene reviewers. 

Changes: The Secretary has added a 
new § 700.21(c)(2), which allows the 
Secretary to use discretion in 
determining whether to convene 
reviewers of unsolicited applications. 
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Comments: One commenter felt that it 
was important that applicants receive 
the written comments regarding the 
strengths and weaknesses of their 
applications at the same time as the 
apphcants are notified of acceptance or 
rejection. Another commenter 
recommended that the OERI standards 
address the issue of OERI's 
communicating the results of the 
competition to the applicants and the 
larger community. 

Discussion: The Secretary recently 
modified Department procedures to 
provide both successful and 
unsuccessful applicants earlier 
notification of funding decisions. In 
most cases, reviewer comments 
accompany these notifications. 

However, particularly for competitions 
that generate laige numbers of 
applications, reviewer comments are 
sent at a later date so that this early 
notification of funding decisions need 
not be delayed. The Secretary routinely 
issues press releases to inform the 
public of the results of each 
competition. 

Changes: None. 

Evaluating Grant and Cooperative 
Agreement Applications and Contract 
Proposals (§ 700,21 and § 700.22) 

Comments: One commenter stated 
that it was not clear that the Secretary 
will be constrained or informed by the 
results of the evaluations carried out by 
the peer reviewers. The commenter 
recommended changes to clarify that: 

(1) Each reviewer will provide a 
recommendation to fund or not to fund 
each application, accompanied by a 
numerical rating of the application; and 

(2) the Secretary will rely on numerical 
ratings given by the peer reviewers in 
rank ordering ihe applications. 

Discussion: The Secretary agrees that 
§ 700.21(e) should be revised to clarify 
that the Secretary prepares a rank order 
based solely on the peer reviewers' 
ratings. However, the Secretary believes 
that, in selecting applications for award, 
he must consider factors in addition to 
the applicants' rankings and the funding 
recommendations of the peer reviewers. 
These other factors, specified in 
§ 700.40, include performance of the 
applicant under a prior award, the 
amount of funds available for the 
competition, and any other information 
relevant to a priority or other statutory 
or regulatory requirement applicable to 
the selection of applications for new 
awards. 

Changes: The Secretary has revised 
§ 700.21(e) to clarify that the rank order 
is based solely on the peer reviewers' 
ratings for each appHcation. 



Comments: One commenter pointed 
out that § 700.22(d), regarding the 
evaluation of contract proposals, 
enables reviewers to assign proposals to 
the category “capable of being made 
acceptable." The commenter 
recommended that a similar category be 
added to § 700.21(d), relating to the 
evaluation of grants and cooperative 
agreements. The commenter believes 
that such a change could allow 
applicants an opportunity to fix minor 
problems in otherwise outstanding grant 
applications and still be eligible for 
funding. 

Discussion: The Federal Acquisition 
Regulations, which govern the Federal 
government's contract procurements 
expressly recommend die establishment 
of a category “capable of being made 
acceptable." In that grant competitions 
are held to determine which applicants 
are to receive the benefit of Federal 
financial assistance for the activities 
applicants propose, rather than to 
determine which applicant or 
applicants will be contracted to provide 
services according to government 
specifications, fairness would dictate 
that if one grant applicant is allowed to 
revise its application, all other grant 
competitors should be provided the 
same opportunity. In addition, grant 
competitions typically generate many 
more apphcations than do contract 
competitions. There are often many 
more highly rated applicants than there 
are funds available for awards, and so 
there is no need to allow competitors a 
second chance to make their 
applications fundable. As a practical 
matter, applications that are truly 
outstanding, but have minor problems, 
are Hkely to be rated highly, with the 
minor problems addressed during 
negotiation of the grant award. The 
Board discussed the issue raised by this 
commenter. The Board was concerned 
that reviewers of proposals for contracts 
had three categories in which they 
could place contract proposals wMle 
reviewers of grant and cooperative 
agreement applications only had two 
categories. The Board recommended 
that a third category be added under 
§ 700.21(d) that would allow reviewers 
to distinguish between projects that they 
would recommend for ftmding and 
those that they would highly 
recommend. 

Changes: The Secretary has added a 
new category of “highly recommended 
for funding" under § 700.21(d). 

Comment: After consultation with the 
Board, the Secretary has determined 
that the evaluation criteria related to 
prior performance of apphcants under 
previously funded grants or cooperative 
agreements (§ 700.30(e)(3)(ii)(D) and 



§ 700.30(e)(4)(ii)(E)) would require 
applicants to provide that information 
for evaluation by the peer reviewers. 
This information is already available to 
the Secretary and will be another factor 
considered by the Secretary in making 
award decisions under § 700.40(a)(3). 

Discussion: The evaluation criteria 
under § 700.30(e)(3)(ii)(D) and 
§ 700.30(e)(4)(ii)(E) dupHcate the factors 
in § 700.40(a)(3) and thus impose an 
imnecessary burden on apphcants. 

Changes: § 700.30(e)(3)(ii)(D) and 
§ 700.30(e)(4)(ii)(E) are deleted. 

Paperwork Reduction Act of 1980 

Section 700.30 contains information 
collection requirements. As required by 
the Paperwork Reduction Act of 1980, 
the Department of Education submitted 
a copy of this section to the Office of 
Management and Budget (OMB) for its 
review, and it was approved by OMB. 
(44 U.S.C. 3504(h)) 

These regulations affect the following 
types of entities ehgible to apply for 
grants and cooperative agreements: State 
or local governments, businesses or 
other for profit organizations, nonprofit 
institutions, and any combinations of 
these types of entities. The Department 
needs and uses the information to 
evaluate apphcations for funding. 

Annual public reporting and 
recordkeeping burden for this collection 
of information is estimated to range 
from 15 hours for each of the 
approximately 750 applications 
expected for a field initiated study 
competition to 150 hours for ten or 
fewer apphcations expected for a 
national research center. Therefore, the 
actual burden will be determined by the 
type of project to be supported in the 
particular competition. 

Intergovernmental Review 

This program is subject to the 
requirements of Executive Order 12372 
and the regulations in 34 CFR Part 79. 
The objective of the Executive order is 
to foster an intergovernmental 
partnership and a strengthened 
federalism by relying on processes 
developed by State and local 
governments for coordination and 
review of proposed Federal financial 
assistance. 

In accordance with the order, this 
document is intended to provide early 
notification of the Department's specific 
plans and actions for this program. 

Assessment of Educational Impact 

Based on the response to the proposed 
regulations and on its own review, the 
Department has determined that the 
regulations in this document do not 
require transmission of information that 
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is boing gathered by or is available from 
any other agency or authority of the 
United States. 

List of Subjects in 34 CFR Part 700 

Education, Educational research, 
Elementary and secondary education. 
Government contracts. Grant 
programs — education. Libraries, 

Reporting and recordkeeping 
requirements. 

Dated: September 11, 1995. 

Richard W. Riley, 

Secretary of Ed ucation . 

Sharon P. Robinson, 

Assistant Secretary for Educational Research 
and Improvement. 

(Catalog of Federal Domestic Assistance 
Number does not apply.) 

The Secretary amends Chapter VII of 
Title 34 of the Code of Federal 
Regulations by adding a new Part 700 to 
read as follows: 

PART 700— STANDARDS FOR THE 
CONDUCT AND EVALUATION OF 
ACTIVITIES CARRIED OUT BY THE 
OFFICE OF EDUCATIONAL 
RESEARCH AND IMPROVEMENT 
(OERI)— EVALUATION OF 
APPLICATIONS FOR GRANTS AND 
COOPERATIVE AGREEMENTS AND 
PROPOSALS FOR CONTRACTS 

Subpart A— General 

Sec. 

700.1 What is the purpose of these 
standards? 

700.2 What activities must be governed by 
these standards? 

700.3 What additional activities may be 
governed by these standards? 

700.4 What definitions apply? 

700.5 What are the processes of open 
competition? 

Subpart B— Selection of Peer Reviewers 

700.10 When is the peer review process 
used? 

700.11 Who may serve as peer reviewers? 

700.12 What constitutes a conflict of 
interest for grants and cooperative 
agreements? 

700.13 What constitutes a conflict of 
interest for contracts? 

Subpart C— The Peer Review Process 

700.20 How many peer reviewers will be 
used? 

700.21 How are applications for grants and 
cooperative agreements evaluated? 

700.22 How are proposals for contracts 
evaluated? 

Subpart D — Evaluation Criteria 

700.30 What evaluation criteria are used for 
grants and cooperative agreements? 

700.31 What additional evaluation criteria 
shall be used for grants and cooperative 
agreements? 

700.32 What evaluation criteria shall be 
used for contracts? 

o 




Subpart E — Selection for Award 

700.40 How are grant and cooperative 
agreement applications selected for 
award? 

700.41 How are contract proposals selected 
for award? 

Authority: 20 U.S.C. 6011(i). 

Subpart A— General 

§ 700.1 What is the purpose of these 
standards? 

(a) The standards in this part 
implement section 91 2 (i) of the 
Educational Research, Development, 
Dissemination, and Improvement Act of 
1994. 

(b) These standards are intended to 
ensure that activities carried out by the 
Office of Educational Research and 
Improvement (the Office) meet the 
highest standards of professional 
excellence. 

(Authority: 20 U.S.C. 6011(i)(l)) 

§700.2 What activities must be governed 
by these standards? 

(a) The standards in this part are 
binding on all activities carried out by 
the Office using funds appropriated 
under section 912(m) of the Educational 
Research, Development, Dissemination, 
and Improvement Act of 1994. 

(b) Activities carried out with funds 
appropriated under section 912(m) of 
the Act include activities carried out by 
the following entities or programs: 

(1) The National Research Institutes. 

(2) The Office of Reform Assistance 
and Dissemination. 

(3) The Educational Resources 
Information Center Clearinghouses. 

(4) The Regional Educational 
Laboratories. 

(5) The Teacher Research 
Dissemination Demonstration Program. 

(6) The Goals 2000 Community 
Partnerships Program. 

(7) The National Educational Research 
Policy and Priorities Board. 

(Authority: 20 U.S.C. 6011(i)(l)) 

§700.3 What additional activities may be 
governed by these standards? 

(a) The Secretary may elect to apply 
the standards in this part to activities 
carried out by the Department using 
funds appropriated under an authority 
other than section 912(m) of the Act. 

(b) (1) If the Secretary elects to apply 
these standards to a competition for new 
grant or cooperative agreement awards, 
the Secretary announces, in a notice 
published in the Federal Register, the 
extent to which these standards are 
applicable to the competition. 

(2) If the Secretary elects to apply 
these standards to a solicitation for a 
contract award, the Secretary announces 



in the request for proposals the extent 
to which these standards are applicable 
to the solicitation. 

(Authority: 20 U.S.C. 6011(i) 

§ 700.4 What definitions apply? 

(a) Definitions in the Educational 
Research, Development, Dissemination, 
and Improvement Act of 1994. The 
following terms used in this part are 
defined in 20 U.S.C. 6011(1): 

Development 

Dissemination 
Educational Research 
Office 

National Research Institute 
Technical Assistance 

(b) Definitions in Education 
Department General Administrative 
Regulations. The following terms used 
in this part are defined in 34 CFR 77.1: 
Applicant 

Application 

Award 

Department 

Grant 

Project 

Secretary 

(c) Definitions in the Federal 
Acquisition Regulation. The following 
terms used in this part are defined in 48 
CFR Chapter 1: 

Contracting Officer 
Employee of an Agency 
Proposal 
Solicitation 

(d) Other definitions. The follovdng 
definitions also apply to this part: 

Act means the Educational Research, 
Development, Dissemination, and 
Improvement Act of 1994 (Title IX of 
Pub. L. 103-227, 108 Stat. 212). 

EDAR means the Education 
Department Acquisition Regulation, 48 
CFR Chapter 34. 

EDGAR means the Education 
Department General Administrative 
Regulations, 34 CFR Parts 74, 75, 77, 79, 
80, 81, 82, 85 and 86. FAR means the 
Federal Acquisition Regulation, 48 CFR 
Chapter 1. 

(Authority: 20 U.S.C. 6011) 

§ 700.5 What are the processes of open 
competition? 

The Secretary uses a process of open 
competition in awarding or entering 
into all grants, cooperative agreements, 
and contracts governed by these 
standards. The processes of open 
competition are the following: 

(a) For all new aweirds for grants and 
cooperative agreements, the Secretary 
vnW make awards pursuant to the 
provisions of EDGAR with the exception 
of the provisions in 34 CFR 75.100(c)(5), 
75.200(b)(3), (b)(5), 75.210, and 
75.217(b)(1), (b)(2), (c), and (d); and 
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(b) For contracts, the Department will 
conduct acquisitions piirsuant to this 
part in accordance with the 
requirements of the Competition in 
Contracting Act, 41 U.S.C. 253, and the 
FAR. 

(Authority: 20 U.S.C. 6011(i)(2); 41 U.S.C. 
253) 

Subpart B— Selection of Peer 
Reviewers 

§700.10 When Is the peer review process 
used? 

The Secretary uses a peer review 
process — 

(a) To review and evaluate all 
applications for grants and cooperative 
agreements and proposals for those 
contracts that exceed $100,000; 

(b) To review and designate 
exemplary and promising programs in 
accordance with section 941(d) of the 
Act; and 

(c) To evaluate and assess the 
performance of all recipients of grants 
from and cooperative agreements and 
contracts with the Office. 

(Authority: 20 U.S.C. 6011(i)(2)(B)) 

§ 700.1 1 Who may sen/e as peer 
reviewers? 

(a) An individual may serve as a peer 
reviewer for purposes of reviewing and 
evaluating applications for new aweirds 
for grants and cooperative agreements 
and contract proposals if the 
individual — 

(1) Possesses the following 
qualifications: 

(1) Demonstrated expertise, including 
training and experience, in the subject 
area of the competition. 

(ii) In-depth loiowledge of policy or 
practice in the field of education. 

(iii) In-depth knowledge of theoretical 
perspectives or methodological 
approaches in the subject area of the 
competition; and 

(2) Does not have a conflict of interest, 
as determined in accordance with 
§700.12. 

(b) For each competition for new 
awards for grants and cooperative 
agreements — 

(1) Department staff may not serve as 
peer reviewers except in exceptional 
circumstances as determined by the 
Secretary; and 

(ii) The majority of reviewers may be 
persons not employed by the Federal 
Government. 

(2) For each review of an imsolicited 
grant or cooperative agreement 
application — 

(i) Department employees may assist 
the Secretary in making an initial 
determination imder 34 CFR 75.222(b); 
and 



o 




(ii) Department employees may not 
serve as peer reviewers in accordance 
with 34 CFR 75.222(c). 

(c) To the extent feasible, the 
Secretary selects peer reviewers for each 
competition who represent a broad 
range of perspectives. 

(Authority: 20 U.S.C. 6011(i)(2)(B)) 

§700.12 What constitutes a conflict of 
interest for grants and cooperative 
agreements? 

(a) Peer reviewers for grants and 
cooperative agreements are considered 
employees of the Department for the 
purposes of conflicts of interest 
analysis. 

(b) As employees of the Department, 
peer reviewers are subject to the 
provisions of 18 U.S.C. 208, 5 CFR 
2635.502, and the Department’s policies 
used to implement those provisions. 

(Authority: 20 U.S.C. 6011(i)(2)(B)) 

§700.13 What constitutes a conflict of 
interest for contracts. 

(a) Peer reviewers for contract 
proposals are considered employees of 
the Department in accordance with 
FAR, 48 CFR 3.104-^(h)(2). 

(b) As employees of the Department, 
peer reviewers are subject to the 
provisions of the FAR, 48 CFR Part 3 
Improper Business Practices and 
Personal Conflict of Interest. 

(Authority: 41 U.S.C. 423) 

Subpart C — ^The Peer Review Process 

§ 700.20 How many peer reviewers will be 
used? 

(a) Each application for a grant or 
cooperative agreement award must be 
reviewed and evaluated by at least three 
peer reviewers except — 

(1) For those grant and cooperative 
agreement awards under $50,000, fewer 
than three peer reviewers may be used 
if the Secretary determines that 
adequate peer review can be obtained 
using fewer reviewers; and 

(2) For those grant and cooperative 
agreement awards of more than 
$1,000,000, at least five reviewers must 
be used. 

(b) Each contract proposal must be 
read by at least three reviewers imless 
the contracting officer determines that 
an adequate peer review can be obtained 
by using fewer reviewers. 

(c) Before releasing contract proposals 
to peer reviewers outside the Federal 
Government, the contracting officer 
shall comply with FAR, 48 CFR 15.413- 
2(f). 

(Authority: 20 U.S.C. 6011(i)(2)(B)) 



§ 700.21 How are applications for grants 
and cooperative agreements evaluated? 

(a) Each peer reviewer must be given 
a number of applications to evaluate. 

(b) Each peer reviewer shall — 

(1) Independently evaluate each 
application; 

(2) Evaluate and rate each application 
based on the reviewer’s assessment of 
the quality of the application according 
to the evaluation criteria and the 
weights assigned to those criteria; and 

(3) Support the rating for each 
application with concise written 
comments based on the reviewer’s 
analysis of the strengths and weaknesses 
of the application with respect to each 
of the applicable evaluation criteria. 

(c) (1) Except as provided in 
paragraph (c)(2) of this section, after 
each peer reviewer has evaluated and 
rated each application independently, 
those reviewers who evaluated a 
common set of applications are 
convened to discuss the strengths and 
weaknesses of those applications. Each 
reviewer may then independently 
reevaluate and re-rate an application 
with appropriate changes made to the 
written comments. 

(2) Reviewers are not convened to 
discuss an imsolicited application 
unless the Secreteiry determines that 
discussion of the application’s strengths 
and weaknesses is necesseiry. 

(d) Following discussion and any 
reevaluation and re-rating, reviewers 
shall independently place each 
application in one of three categories, 
either “highly recommended for 
funding,’’ “recommended for funding’’ 
or “not recommended for funding.’’ 

(e) After the peer reviewers have 
evaluated, rated, and made funding 
recommendations regarding the 
applications, the Secreteuy prepares a 
rank order of the applications based 
solely on the peer reviewers’ ratings. 

(Authority: 20 U.S.C. 6011(i)(2)(C)) 

§ 700.22 How are proposals for contracts 
evaluated? 

(a) Each peer reviewer must be given 
a number of technical proposals to 
evaluate. 

(b) Each peer reviewer shall — 

(1) Independently evaluate each 
technical proposal; 

(2) Evaluate and rate each proposal 
based on the reviewer’s assessment of 
the quality of the proposal according to 
the technical evaluation criteria and the 
importance or weight assigned to those 
criteria; and 

(3) Support the rating for each 
proposal with concise written 
comments based on the reviewer’s 
analysis of the strengths and weaknesses 
of the proposal with respect to each of 
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the applicable technical evaluation 
criteria. 

(c) After each peer reviewer has 
evaluated each proposal independently, 
those reviewers who evaluated a 
common set of proposals may be 
convened to discuss the strengths and 
weaknesses of those proposals. Each 
reviewer may then independently 
reevaluate and re-rate a proposal with 
appropriate changes made to the written 
comments. 

(d) Following discussion and any 
reevaluation and re-rating, reviewers 
shall rank proposals and advise the 
contracting officer of each proposal's 
acceptabihty for contract award as 
“acceptable," “capable of being made 
acceptable without major 
modifications," or “unacceptable." 
Reviewers may also submit technical 
questions to be asked of the offeror 
regarding the proposal. 

(Authority: 20 U.S.C. 6011(i)(2KC)) 

Subpart D — Evaluation Criteria 

§ 700.30 What evaluation criteria are used 
for grants and cooperative agreements? 

(a) Except as provided in paragraph 
(d) of this section, the Secretary 
announces the applicable evaluation 
criteria for each competition and the 
assigned weights in a notice published 
in the Federal Register or in the 
application package. 

(b) In determining the evaluation 
criteria to be used in each grant and 
cooperative agreement competition, the 
Secretary selects firom among the 
evaluation criteria in paragraph (e) of 
this section and may select firom among 
the specific factors listed under each 
criterion. 

(c) The Secretary assigns relative 
weights to each selected criterion and 
factor. 

(d) In determining the evaluation 
criteria to be used for unsolicited 
applications, the Secretary selects firom 
among the evaluation criteria in 
paragraph (e) of this section, and may 
select firom among the specific factors 
listed under each criterion, the criteria 
which are most appropriate to evaluate 
the activities proposed in the 
application. 

(e) The Secretary establishes the 
following evaluation criteria: 

(1) National significance. 

(i) The Secretary considers the 
national significance of the proposed 
project. 

(ii) In determining the national 
significance of the proposed project, the 
Secretary may consider one or more of 
the following factors: 

(A) The importance of the problem or 
issue to be addressed. 



(B) The potential contribution of the 
project to increased knowledge or 
understanding of educational problems, 
issues, or effective strategies. 

(C) The scope of the project. 

(D) The potential for generalizing 
from project findings or results. 

(E) The potential contribution of the 
project to the development and 
advancement of theory and knowledge 
in the field of study. 

(F) Whether the project involves the 
development or demonstration of 
creative or innovative strategies that 
build on, or are alternatives to, existing 
strategies. 

(G) The nature of the products (such 
as information, materials, processes, or 
techniques) likely to result firom the 
project and the potential for their 
effective use in a variety of other 
settings. 

(H) The extent and quality of plans for 
disseminating results in ways that will 
allow others to use the information. 

(2) Quality of the project design. 

(i) The Secretary considers the quality 
of the design of the proposed project. 

(ii) In determining the quality of the 
design of the proposed project, the 
Secretary may consider one or more of 
the following factors: 

(A) Whether the goals, objectives, and 
outcomes to be achieved by the project 
are clearly specified and measurable. 

(B) Whether there is a conceptual 
firamework underlying the proposed 
activities and the quality of that 
firamework. 

(C) Whether the proposed activities 
constitute a coherent, sustained program 
of research and development in the 
field, including a substantial addition to 
an ongoing line of inquiry. 

(D) Whether a specific research design 
has been proposed, and the quality and 
appropriateness of that design, 
including the scientific rigor of the 
studies involved. 

(E) The extent to which the research 
design includes a thorough, high-quality 
review of the relevant fiterature, a high- 
quality plan for research activities, and 
the use of appropriate theoretical and 
methodological tools, including those of 
a variety of disciplines, where 
appropriate. 

(F) The quality of the demonstration 
design and procedures for documenting 
project activities and results. 

(G) The extent to which development 
efforts include iterative testing of 
products and adequate quahty controls. 

(H) The likelihood that the design of 
the project will successfully address the 
intended, demonstrated educational 
need or needs. 

(I) How well and innovatively the 
project addresses statutory purposes. 



requirements, and any priority or 
priorities announced for the program. 

(J) The quality of the plan for 
evaluating the functioning and impact 
of the project, including the objectivity 
of the evaluation and the extent to 
which the methods of evaluation are 
appropriate to the goals, objectives, and 
outcomes of the project. 

(3) Quality ana potential 
contributions of personnel. 

(i) The Secretary considers the quality 
and potential contributions of personnel 
for the proposed project. 

(ii) In determining the quality and 
potential contributions of personnel for 
the proposed project, the Secretary may 
consider one or more of the following 
factors: 

(A) The qualifications, including 
training and experience, of the project 
director or principal investieator. 

(B) The qualifications, inmuding 
training and experience, of key project 
personnel. 

(C) The qualifications, including 
training and experience, of proposed 
consultants or subcontractors. 

(4) Adequacy of resources. 

(i) The Secretary considers the 
adequacy of resources for the proposed 
project. 

(ii) In determining the adequacy of 
resources for the proposed project, the 
Secretary may consider one or more of 
the following factors: 

(A) The adequacy of support from the 
lead applicant organization. 

(B) The relevance and commitment of 
each partner in the project to the 
implementation and success of the 
project. 

(C) Whether the budget is adequate to 
support the project. 

(D) Whether the costs are reasonable 
in relation to the objectives, design, and 
potential significance of the project. 

(E) The potential for continued 
support of the project after federal 
funding ends. 

(5) Quality of the management plan. 

(i) The Secretary considers the quality 
of the management plan of the proposed 
project. 

(ii) In determining the quality of the 
management plan of a proposed project, 
the Secretary may consider one or more 
of the following factors: 

(A) The adequacy of the management 
plan to achieve the objectives of the 
project, including the specification of 
staff responsibihty, timelines, and 
benchmarks for accomplishing project 
tasks. 

(B) The adequacy of plans for 
ensuring high-quality products and 
services. 

(C) The adequacy of plans for 
ensuring continuous improvement in 
the operation of the project. 
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(D) Whether time commitments of the 
project director or principal investigator 
and other key personnel are appropriate 
and adequate to meet project objectives. 

(E) How the applicant will ensure that 
a diversity of perspectives are brought to 
bear in the operation of the project, 
including those of parents and teachers, 
where appropriate. 

(F) How the applicant will ensure that 
persons who are otherwise eligible to 
participate in the project are selected 
without regard to race, color, national 
origin, gender, age, or disability. 

(G) The adequacy of plans for 
widespread dissemination of project 
results and products in ways that will 
assist others to use the information. 

(Approved by the Office of Management and 
Budget under control number 1850-0723) . 
(Authority: 20 U.S.C. 6011(i)(2)(D)(ii)) 

§ 700.31 What additional evaluation 
criteria shall be used for grants and 
cooperative agreements? 

In addition to the evaluation criteria 
established in § 700.30(e), the Secretary 
uses criteria or factors specified in the 
applicable program statute to evaluate 
applications for grants and cooperative 
agreements. 

(Authority: 20 U.S.C. 601l(i)(2)(D)(ii)) 







§700.32 What evaluation criteria shall be 
used for contracts? 

(a) The evaluation criteria to be 
considered in the technical evaluation 
of contract proposals are contained in 
the FAR at 48 CFR 15.605. The 
evaluation criteria that apply to an 
acquisition and the relative importance 
of those factors are within the broad 
discretion of agency acquisition 
officials. 

(b) At a minimum, the evaluation 
criteria to be considered must include 
cost or price and quality. Evaluation 
factors related to quality are called 
technical evaluation criteria. 

(c) Technical evaluation criteria may 
include, but are not limited to, the 
followine: 

(1) Teomical excellence. 

(2) Management capability. 

(3) Personnel qualifications. 

(4) Prior experience. 

(5) Past performance. 

(6) Schedule compliance. 

(Authority: 20 U.S.C. 6011(i)(2)(D)(ii)) 

Subpan E — Selection for Award 

§ 700.40 How are grant and cooperative 
agreement applications selected for award? 

(a) The Secretary determines the order 
in which applications will be selected 
for grants and cooperative agreement 
awards. The Secretary considers the 



following in making these 
determinations: 

(1) An applicant’s ranking. 

(2) Recommendations of the peer 
reviewers with regard to funding or not 
funding. 

(3) Information concerning an 
applicant’s performance and use of 
funds \mder a previous Federal award. 

(4) Amount of funds available for the 
competition. 

(5) Any other information relevant to 
a priority or other statutory or regulatory 
requirement applicable to the selection 
of applications for new awards. 

(b) In the case of unsolicited 
applications, the Secretary uses the 
procedures in EDGAR (34 CFR 75.222(d) 
and (e)). 

(Authority: 20 U.S.C. 6022(i)(2)(D)(i)) 

§ 700.41 How are contract proposals 
selected for award? 

Following evaluation of the proposals, 
the contracting officer shall select for 
award the offeror whose proposal is 
most advantageous to the Government 
considering cost or price and the other 
factors included in the solicitation. 
(Authority: 20 U.S.C. 6011(i)(2)(D)(i)) 

[FR Doc. 95-22872 Filed 9-13-95; 8:45 am) 
eiLUNQ CODE 4(KKM>1-P 
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DEPARTMENT OF EDUCATION 

34 CFR Part 701 

RIN 1850-AA52 

Standards for Conduct and Evaluation 
of Activities Carried out by the Office 
of Educationai Research and 
improvement (OERi) — Designation of 
Exempiary and Promising Programs 

agency: Department of Education. 
ACTION: Final regulation. 

summary: The Assistant Secretary for 
Educational Research and Improvement 
(OERI) is establishing final regulations 
pursuant to the “Educational Research, 
Development, Dissemination, and 
Improvement Act of 1994.” The 
regulations are intended to provide 
quality assurance that programs 
designated by the Department of 
Education as either exemplary or 
promising have met criteria that will 
allow educators, professional 
organizations, and others to use these 
programs with confidence. 
dates: These regulations take effect 
December 17, 1997. However, affected 
parties do not have to comply with the 
information collection requirement in 
§ 701.4 until the Department of 
Education publishes in the Federal 
Register notification of the compliance 
date and the control number assigned by 
the Office of Management and Budget 
(OMB) to this information collection 
requirement. Publication of the control 
number notifies the public that OMB 
has approved this information 
collection requirement under the 
Paperwork Reduction Act of 1995. 

FOR FURTHER INFORMATION CONTACT: 
Stephen O^Brien, U.S. Department of 
Education, 555 New Jersey Avenue, 

NW, Room 502B, Washington, D.C. 
Telephone: (202) 219-2141. Internet: 

(Steve 0’Brien@ed.gov). Individuals 

who use a telecommimication device for 
the deaf (TDD) may call the Federal 
Information Relay Service (FIRS) at 1- 
800-877-8339 between 8 a.m. and 8 
p.m.. Eastern time, Monday through 
Friday. 

Individuals with disabilities may 
obtain this document in an alternate 
format (e.g., Braille, large print, 
audiotape, or computer diskette) on 
request to the person listed in the 
preceding paragraph. 

SUPPLEMENTARY INFORMATION: On March 
31, 1994, President Clinton signed 
Public Law 103-227, which includes 
Title IX, the Educational Research, 
Development, Dissemination, and 
Improvement Act of 1994 (the Act). The 
Act restructured OERI and provided it 



with a broad mandate to conduct an 
array of research, development, 
dissemination, and improvement 
activities aimed at strengthening the 
education of all students. 

The Act directed the Assistant 
Secretary to develop, in consultation 
with the National Educational Research 
Policy and Priorities Board (the Board), 
the highest standards of professional 
excellence necessary to govern the 
conduct and evaluation of all research, 
development, and dissemination 
activities carried out by the OERI. The 
legislation requires that the standards be 
developed in three phases. 

In the first phase, standards were 
promulgated to establish the peer 
review process and evaluation criteria to 
be used for reviewing applications for 
grants and cooperative agreements and 
proposals for contracts. The Department 
published final regulations setting out 
these standards on September 14, 1995 
(60 FR 47808). The regulations in this 
announcement address the second 
phase of development by establishing 
the criteria for panels to use in 
reviewing potentially exemplary and 
promising educational programs. The 
Assistant Secretary will later publish 
proposed regulations for phase three of 
the standards, which will govern how 
OERI evaluates performance of its 
recipients of grants, contracts, and 
cooperative agreements. 

Tne OERI legislation requires that 
expert panels be established to review 
educational programs submitted by 
individuals or organizations. The 
legislation also provides that the 
Secretary may identify educational 
programs for the panels to review. The 
statute requires the panels to 
recommend to the Secretary those 
programs that should be designated as 
exemplary or promising and 
disseminated through the Department’s 
National Education Dissemination 
System. The law requires that each 
panel consist of appropriately qualified 
experts and practitioners and requires 
the Secretary to develop standards that 
describe the procedures the panels will 
use in reviewing the educational 
programs. Section 941(a)(3) of the law 
broadly defines educational programs to 
include educational polices, research 
findings, practices, and products. 
Educational programs may range in size 
and complexity from an individual 
instructional program — such as an 
elementary school science program — to 
a comprehensive reform initiative 
involving multiple goals and 
participants. Programs at all levels of 
education — preschool, elementary, 
secondary, and postsecondary — Eire 
eligible for consideration. 



In determining whether an 
educational program should be 
recommended as exemplary or 
promising, each panel is required by the 
Act to consider: (a) Whether, based on 
empirical data, the program is effective 
and should be designated as exemplary 
or (b) whether there is sufficient 
evidence to demonstrate that the 
program shows promise for improving 
student achievement and should be 
designated as promising. The Act 
expressly states that a panel shall not 
eliminate a program from consideration 
based on the lack of one type of 
supporting data such as test scores. 

The evaluation process set forth in the 
final regulations will ensure that 
programs disseminated by the 
Department are high-quality, research- 
based programs that have provided 
evidence indicating they have improved 
teaching, learning, or both, or has 
demonstrated other worthy educational 
performance outcomes. The 
Department’s dissemination system is 
designed to make information about 
these promising and exemplary 
programs available to the public as 
quickly as possible. The system will 
enable the Department to respond to all 
forms of requests for information and 
assistance, and to support the 
applications of research and best 
practice. The system will use electronic 
networking and the capabilities of: 

— National Research Institutes; 

— Educational Resources Information 
Center (ERIC); 

— Regional Educational Laboratories; 

— Department-supported dissemination 
and technical assistance providers; 

— National Library of Education; 

— Eisenhower Regional Consortia and 
Clearinghouse, and 
— Other public and private nonprofit 
entities, including education 
associations and networks. 

Until recently, the Department 
validated exemplary programs through 
its Program Effectiveness Panel (PEP) 
and disseminated them through the 
National Diffusion Network (NDN). 
Since this program no longer exists, 
with the adoption of these standards the 
Department will evaluate and 
disseminate promising educational 
programs in addition to exemplary 
programs. The Department will also 
work in partnership with constituency 
groups who have expertise in the 
specific topic areas represented by the 
expert panels to develop coordinated 
procedures to maximize their 
involvement in this work. 

On Jime 3, 1996, the Secretary 
published a notice of proposed 
rulemaking (NPRM) for this part in the 
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Federal Register (61 FR 27990-27993). 
These final regulations contain three 
major changes from the NPRM. These 
changes are fully explained in the 
“Analysis of Comments and Changes" 
elsewhere in this preamble. The changes 
pertain to the standing panel; the 
distinction between “promising" and 
“exemplary"; and the factors listed 
under the criteria expert panels will use 
to evaluate programs. 

Analysis of Comments and Changes 

In response to the Secretary's 
invitation in the NPRM, seven parties 
submitted comments on the proposed 
regulations. This included comments 
from individual members of two pilot 
panels (math/science and gender equity) 
that were appointed by the Secretary to 
field test the expert panel process. In 
addition to the public comment, 
comments from the Board's 
Subcommittee on Standards are 
addressed as required by the legislation. 
The full Board approved the final 
regulations at a meeting on September 
26, 1997. An analysis of the comments 
and of the changes in the regulations 
since publication of the NPRM follows. 

Major issues are grouped according to 
subject with appropriate sections of the 
regulations referenced in parentheses. 
Technical and other minor changes — 
and suggested changes the Secretary is 
not legally authorized to make under the 
applicable statutory authority — are not 
addressed. 

Eligibility (§701.3) 

Comments: One commenter asked for 
clarification on who is eligible to submit 
educational programs for designation as 
promising or exemplary. Specifically, 
this commenter asked whether 
federally-funded entities, such as the 
Regional Laboratories, will be required 
to go through this process; whether local 
agencies that receive Federal funding 
through states, such as under Title I of 
the Elementary and Secondary 
Education Act (ESEA), may submit 
programs on their own; whether 
sponsors need to be invited to submit or 
may submit on their own initiative; and 
whether for-profit entities may submit. 

Discussion: The law provides that 
“individuals" or “organizations" may 
submit educational programs for review. 
Since the law is silent on the specific 
nature of the organizations, the 
Secretary believes that for-profit 
agencies would be eligible to submit 
programs for review. With respect to the 
OERI-funded Regional Educational 
Laboratories, the law provides that the 
Secretary may identify those programs 
for panel review. In addition, the 
Secretary believes that the Laboratories 

O 




could submit one or more of their 
programs on their own initiative. The 
question of whether local agencies that 
receive Federal funding through a State 
or Federal entity, such as under Title I 
of the ESEA, can submit on their own 
or must go through their funding 
agency, will be addressed in 
administrative guidance. 

Changes: None. 

Content of Submissions (§ 701 .4 ) 

Comments: Three commenters made 
suggestions about this section. Two 
commenters believed that requiring 
funding and staffing information was 
burdensome and not germane to the 
designation of a program as promising 
or exemplary. One commenter believed 
that this section should require program 
sponsors to submit specific materials 
related to content and methods. Another 
commenter believed that this section 
should include the requirement that the 
program include evidence of 
sustainability of improvement with 
targeted student populations. 

Discussion: The Secretary believes 
that funding and staffing information 
should be included to help determine 
whether an educational program should 
be recommended as either exemplary or 
promising. The Secretary agrees that 
sponsors should be required to submit 
information or materials specific to 
content and methods, as available and 
appropriate. The Secretary believes that 
the evidence of sustainability of student 
improvement should be evaluated by 
peer reviewers in accordance with 
§701.22. 

Changes: Section 701.4(b)(7) has been 
renumbered as § 701.4(b)(8) and a new 
§ 701.4(b)(7) has been added to include 
a provision for specific materials 
relevant to content and methods. 

Procedures for Submitting Educational 
Programs (New §701.5) 

Comments: One commenter believed 
that the regulations should contain more 
specificity about the procedure for 
submitting programs to the expert 
panels. This commenter requested 
specifics on who receives the 
submissions and whether they may be 
ysubmitted at any time or only on 
specific dates. 

Discussion: The Secretary agrees that 
the general submission procedures 
should be included in regulations. A 
sponsor seeking the exemplary or 
promising designation for its 
educational program may submit its 
program at any time for consideration to 
the Assistant Secretary, who will assign 
the submitted program to the 
appropriate panel for review. The 
individual expert panels will set 



appropriate timelines for program 
submissions. In addition, the Assistant 
Secretary will periodically establish and 
announce in the Federal Register 
specific topic areas of high priority. 
Sponsors of educational programs in 
these areas will be invited to submit 
them for consideration. 

Changes: A new § 701.5 has been 
added to include general procedures for 
submitting educational programs for 
review by an expert panel. 

Establishment of Panels (§ 701.10) 

Comments: The Board's 
Subcommittee on Standards 
recommended a change to the expert 
panel system. The Subcommittee 
thought that the structure of having 
members of the expert panels drawn 
from a separate standing panel of 
educational experts was an unwieldy, 
overly-complicated structure. The Board 
recommended that the expert panels be 
formed separately from a standing 
panel, which would instead provide an 
administrative oversight and monitoring 
function for the expert panels. 

Discussion: The Secretary agrees that 
the expert panel should be formed 
separately from a standing panel. The 
Secretary will determine the feasibility 
of establishing a separate standing panel 
for the oversight and monitoring 
functions referred to by the Board — 
functions which are administrative in 
nature and could also be performed by 
OERI staff. Elimination of a reference to 
a standing panel in the regulations 
would not alter the composition and 
function of the expert panels as outlined 
in the NPRM. 

Changes: Section 701.10(a) has been 
removed, § 701.10(b) has been revised, 

§ 701.11 has been removed, § 701.12(a) 
has been revised, and § 701.12 has also 
been renumbered as § 701.11. 

Panel Membership (§§ 701.11 and 
701.12) 

Comments: One commenter observed 
that §§ 701.11 and 701.12 in the NPRM 
did not explicitly state that those 
serving on the panels would represent 
both the community of practice and that 
of research. One commenter believed 
that each panel should include one or 
more members with evaluation 
expertise in order to help evaluate 
evidence of effectiveness. 

Discussion: The Secretary agrees with 
these comments. 

Changes: A sentence has been added 
at the end of the renumbered § 701.11(a) 
(formerly § 701.12(a)) stating that the 
membership of the expert panels will 
represent both the community of 
practice and the community of research. 
Additionally, §§ 701.11(b)(3) and 
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701.11(b)(4) have been renumbered as 
§§ 701.11(b)(4) and 701.11(b)(5), 
respectively; and a new provision for 
§ 701.11(b)(3) has been added to include 
the selection of at least one individual 
with expertise in evaluating educational 
programs. 

Difference Between Promising and 
Exemplary Programs (§ 701.21) 

Comments: Five comments were 
received on the distinction between 
promising and exemplary programs. As 
proposed in § 701.21, the distinction 
was based upon the generalizability of 
the educational programs. Promising 
programs had to meet each of the 
criteria of educational effectiveness in 
§ 701.22 (success, quality, educational 
significance, and usefulness to others) 
with respect to only one ‘'context or 
population.'' Exemplary programs had 
to meet each of the criteria “with 
respect to multiple contexts or multiple 
populations.'' 

Two commenters believed that the 
distinction should stay the way it was 
in the NPRM, although one of those 
suggested some clarifying language. 
However, three commenters questioned 
the distinction on the basis that it was 
too narrowly and artificially drawn and 
did not reflect the commonly 
understood meaning of the words 
“promising" and “exemplary." In this 
regard, one commenter believed that 
promising programs should not have to 
meet every criterion in § 701.22 at the 
same level as exemplary programs. Two 
commenters believed that promising 
programs should have to meet the 
criteria at the same level as exemplary, 
hut that the evidence required of 
promising programs should be less 
stringent and that exemplary programs 
should be held to a higher standard of 
evidence. 

Discussion: The Secretary agrees with 
those commenters who questioned the 
proposed distinction and advised OERI 
to give a more common sense meaning 
to the terms “promising" and 
“exemplary." The Secretary believes 
that the distinction between promising 
and exemplary programs specified in 
legislation is sufficient to cover these 
concerns. The Secretary relies upon the 
expert judgment of the expert panel 
reviewers in determining the nature and 
weight of evidence necessary to 
designate a program as either promising 
or exemplary, and in applying the 
criteria listed in § 701.22 in making this 
determination. 

Changes: A revision has heen made to 
the distinction between “promising" 
and “exemplary." 

O 




Criteria (§701.22) 

Comments: Five commenters 
provided comments on this section and 
suggested revisions to either the 
wording of the criteria or to the content 
of the factors listed under each criterion 
or both. These comments included 
comments from one member of the 
math/science pilot panel and three 
members of the gender equity pilot 
panel. Although the math/science panel 
member did provide comments specific 
to the proposed criteria and factors, the 
consensus of this panel was that the 
expert panel process would be better 
served if each panel developed its own 
factors specific to the content or 
discipline or both under review by the 
individual panel. One commenter 
suggested that the word “replicability" 
would better capture the concept for the 
criterion entitled “usefulness to others." 
In addition, OERI's Board (The 
Subcommittee on Standards) thought 
that the regulations should be as simple 
as possible and should give the expert 
panels as much discretion as possible in 
evaluating programs submitted for 
review. 

Discussion: In addition to the math/ 
science and gender equity panels, the 
Secretary will establish pilot panels in 
technology and early reading in the next 
year. The Secretary has determined that 
until the work of all four pilot panels is 
concluded, the regulation should retain 
only the four criteria outlined in the 
NPRM in § 701.22 and allow each panel 
the flexibility to establish its own 
individual factors under each criterion 
that are specific to its content or 
discipline. The fact that the comments 
from the public suggested various 
changes to the factors underscores the 
desirability of this approach. While the 
final regulations will therefore no longer 
require the expert panels to apply the 
factors listed in the NPRM, the Secreteiry 
encourages each panel to look at these 
factors as suggested examples. The 
Secretary will review the factors 
developed by all of the panels to see if 
the criteria set forth in the final 
regulations need to be modified. 

Changes: The factors specified under 
each of the four criteria have been 
eliminated and the criterion, 

“usefulness to others" has been changed 
to “replicability." 

Assessment of Educational Impact 

In the NPRM the Secretary requested 
comments on whether the proposed 
regulations would require transmission 
of information that is being gathered by 
or is available from any other agency or 
authority of the United States. 



Based on the response to the NPRM 
and on its own review, the Department 
has determined that the regulations in 
this document do not require 
transmission of information that is being 
gathered by or is available from any 
other agency or authority of the United 
States. 

Electronic Access to This Document 

Anyone may view this document, as 
well as all other Department of 
Education documents published in the 
Federal Register, in text or portable 
document format (pdf) on the World 
Wide Web at either of the following 
sites: 

http ://ocfo. ed.gov/ fedreg.htm 
http :/ / www.ed.gov/news.html 
To use the pdf you must have the 
Adobe Acrobat Reader Program with 
Search, which is available free at either 
of the previous sites. If you have 
questions about using the pdf, call the 
U.S. Government Printing Office toll 
free at 1-888-293--6498. 

Anyone may also view these 
documents in text copy only on an 
electronic bulletin board of the 
Department. Telephone: (202) 219—1511 
or, toll free, 1-800-222-4922. The 
documents are located under Option 
G — Files/ Announcements, Bulletins and 
Press Releases. 

Note: The official version of this document 
is the document published in the Federal 
Register. 

List of Subjects in 34 CFR Part 701 

Education, Educational research. 
Reporting and recordkeeping 
requirements. 

(Catalog of Federal Domestic Assistance 
Number does not apply) 

Dated: November 11, 1997. 

Ricky T. Takai, 

Acting Assistant Secretary for Educational 
Research and Improvement. 

The SecretEiry amends chapter VII of 
title 34 of the Code of Federal 
Regulations by adding a new part 701 to 
read as follows: 

PART 701— STANDARDS FOR 
CONDUCT AND EVALUATION OF 
ACTIVITIES CARRIED OUT BY THE 
OFFICE OF EDUCATIONAL 
RESEARCH AND IMPROVEMENT 
(OERI)— DESIGNATION OF 
EXEMPLARY AND PROMISING 
PROGRAMS 

Subpart A — General 

Sec. 

701.1 What is the purpose of these 
standards? 

701.2 What definitions apply? 

701.3 Who is eligible to submit an 
educational program for review? 
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701.4 What must a program sponsor submit 
for review? 

701.5 What are the procedures for 
submitting an educational program for . 
review by an expert panel? 

Subpart B — Selection of Panel Members 

701.10 How are panels established? 

701.11 How is the membership of expert 
panels determined? 

Subpart C — ^The Expert Panel Review 
Process 

701.20 How does an expert panel evaluate 
programs? 

701.21 What is the difference between an 
exemplary and a promising program? 

701.22 What criteria are used to evaluate 
programs for exemplary or promising 
designation? 

Authority: 20 U.S.C. 6011(i), unless 
otherwise noted. 

Subpart A— General 

§701.1 What Is the purpose of these 
standards? 

(a) The standards in this part 
implement section 941(d) of the 
Educational Research, Development, 
Dissemination, and Improvement Act of 
1994. 

(h) These standards are intended to 
provide quality assurance that 
educational programs designated by the 
U.S. Department of Education as either 
exemplary or promising have met 
criteria that will allow educators, 
professional organizations, and others to 
use these programs with confidence. 

(Authority: 20 U.S.C. 6011(i)(2)(B)(iii) and 
(E), 6041(d)) 

§ 701 .2 What definitions apply? 

The following definitions apply to 
this part: 

Assistant Secretary means the 
Assistant Secretary for the Office of 
Educational Research and Improvement. 

Educational programs mean 
educational policies, research findings, 
practices, and products. 

Program sponsor means a party 
submitting an educational program for 
designation by the Secretary as either 
promising or exemplary. 

Secretary means the Secretary of the 
Department of Education or an official 
or employee of the Department acting 
for the Secretary under a delegation of 
authority. 

(Authority: 20 U.S.C. 6011(i)(2)(B)(iii) and 
(E), 6041(d)) 

§ 701 .3 Who Is eligible to submit an 
educational program for review? 

Any public or private agency, 
organization or institution, or an 
individual may submit an educational 
program for review. 

O 




(Authority: 20 U.S.C. 6011(i)(2)(B)(iii) and 
(E), 6041(d)) 

§ 701.4 What must a program sponsor 
submit for review? 

(а) To have an educational program 
considered for designation as exemplary 
or promising, a sponsor must submit to 
the Secretary a description of the 
program, program materials, and a 
discussion of the program that is 
re^onsive to the criteria in § 701.22. 

(d) Information submitted must 
include, to the extent relevant to the 
particular program, — 

(1) A program abstract of 250 words 
or less; 

(2) A description of the salient 
features of the program; 

(3) A description of the program’s 
philosophy and history; 

(4) Site information, including 
demographics; 

(5) A description of evaluation results; 

(б) Funding and staffing information; 

(7) Specific materials relevant to 
content and methods, as appropriate; 
and 

(8) Organization name, address, 
telephone and fax numbers, e-mail 
address (if available), and contact 
person. 

(Authority: 20 U.S.C. 6011(i)(2)(B)(iii) and 
(E), 6041(d)) 

§ 701 .5 What are the procedures for 
submitting an educational program for 
review by an expert panel? 

(a) An applicant seeking the 
exemplary or promising designation for 
its educational program may submit its 
program at any time for consideration to 
the Assistant Secretary, who will assign 
the submitted program to the 
appropriate expert panel for review. 

(b) The Assistant Secretary will 
periodically establish and announce in 
the Federal Register specific topic areas 
of high priority. Sponsors of educational 
programs in these areas will be invited 
to submit their programs for 
consideration. 

(c) The individual expert panels will 
set appropriate timelines for reviewing 
program submissions. 

(Authority: 20 U.S.C. 6011(i)(2)(B)(iii) and 
(E), 6041(d)) 

Subpart B — Selection of Panel 
Members 

§701.10 How are panels established? 

The Assistant Secretary selects 
individuals, based on their areas of 
expertise, to serve on expert panels in 
specific topic areas for the purpose of 
reviewing and evaluating educational 
programs and recommending, to the 
Secretary, those programs that should be 
designated as exemplary or promising. 



(Authority: 20 U.S.C. 6011(i)(2)(B)(iii) and 
(E), 6041(d)) 

§ 701 .1 1 How is the membership of expert 
panels determined? 

(a) For the review of each progreim or 
group of programs, the Assistant 
Secretary establishes an expert panel. 
The membership of the expert panels 
will represent both the community of 
practice and the community of research. 

(b) In establishing the membership of 
each expert panel, the Assistant 
Secretary — 

(1) Selects individuals who have in- 
depth knowledge of the subject area or 
content of the program or group of 
programs to be evaluated; 

(2) Selects at least one current teacher, 
principal, or other school-based or 
community-based professional; 

(3) Selects at least one individual with 
expertise in evaluating educational 
programs; 

(4) Ensures that no more than one- 
third of the panel members are 
employees of the Federal Government; 
and 

(5) Ensures that each panel member 
does not have a conflict of interest, as 
determined in accordance with 
p£uagraph (c) of this section, with 
respect to any educational program the 
panel member is asked to review. 

(c) Panel members are considered 
employees of the U.S. Department for 
the purposes of conflicts of interest 
analysis and are subject to the 
provisions of 18 U.S.C. 208, 5 CFR 
2635.502, and the Department’s policies 
used to implement those provisions. 

(Authority: 20 U.S.C. 6011(i)(2)(B)(iii) and 
(E), 6041(d)) 

Subpart C— The Expert Panel Review 
Process 

§ 701.20 How does an expert panel 
evaluate programs? 

(a) Each panel member shall — 

(1) Independently review each 
program based on the criteria in 
§701.22; 

(2) Provide vrritten comments based 
on an analysis of the strengths and 
weaknesses of the program according to 
the criteria; 

(3) Participate in site visits or other 
verification activities, if appropriate; 
and 

(4) Participate in a meeting of the 
expert panel, if appropriate, to discuss 
the reviews. 

(b) A panel may not eliminate an 
educational program from consideration 
based solely on the fact that the program 
does not have one specific type of 
supporting data, such as test scores. 

(cj Each expert panel shall make a 
recommendation to the Secretary as to 
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whether the program is exemplary, 
promising, or neither. 

(Authority: 20 U.S.C. 6011(i)(2)(B)(iii) and 
(E). 6041(d)) 

§ 701.21 What Is the difference between an 
exemplary and a promising program? 

(a) In determining whether an 
educational program should be 
recommended as exemplary or 
promising, the panel shall consider — 

(1) Whether, based on empirical data, 
the program is effective and should be 
designated as exemplary; or 

(2) Whether there is sufficient 
evidence to demonstrate that the 
program shows promise for improving 



o 




student achievement and should be 
designated as promising. 

(b) The Secretary relies upon the 
judgment and expertise of peer 
reviewers, as established in § 701.11, to 
determine the nature and extent of 
evidence required to distinguish 
between promising and exemplary 
programs and to apply the four criteria 
established in § 701.22, and their own 
individual factors under each criterion 
in making this determination. 

(Authority: 20 U.S.C. 6011(i)(2)(B)(iii) and 
(E), 6041(d)) 



§ 701 .22 What criteria are used to evaluate 
programs for exemplary or promising 
designation? 

The Secretary establishes the 
following evaluation criteria for expert 
panels to use in determining whether an 
educational program should be 
recommended as exemplary, promising, 
or neither: 

(a) Evidence of success. 

(b) Quality of the program. 

(c) Educational significance. 

(d) Replicability. 

(Authority: 20 U.S.C. 6011(i)(2)(B)(iii) and 
(E), 6041(d)) 

[FR Doc. 97-30051 Filed 11-14-97; 8:45 am] 
BILLING CODE 400(M)1>P 
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Part VII — Department of Education 
34 CFR Part 702 

Standards for Conduct and Evaluation 
of Activities Carried Out by the Office 
of Educational Research and 
Improvement (OERI) — Evaluation of 
the Performance of Recipients of 
Grants, Cooperative Agreements, and 
Contracts; Final Rule 
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34 CFR Part 702 

Standards for Conduct and Evaluation of 
Activities Carried Out by the Office of 
Educational Research and Improvement 
(OERI) — Evaluation of the Performance of 
Recipients of Grants, Cooperative 
Agreements, and Contracts; Final Rule 
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DEPARTMENT OF EDUCATION 

34 CFR Part 702 

RIN 1850-AA54 

Standards for Conduct and Evaluation 
of Activities Carried Out by the Office 
of Educational Research and 
Improvement (OERI)— Evaluation of 
the Performance of Recipients of 
Grants, Cooperative Agreements, and 
Contracts 

AGENCY: Office of Educational Research 
and Improvement, Department of 
Education. 

ACTION: Final regulations. 

SUMMARY: The Assistant Secretary 
establishes regulations pursuant to 
OERI’s authorizing legislation, the 
Educational Research, Development, 
Dissemination, and Improvement Act of 
1994. The major purpose of these 
standards is to ensure that the research, 
development, and dissemination 
activities carried out by the recipients of 
grants from and contracts and 
cooperative agreements with OERI meet 
the highest standards of professional 
excellence. 

EFFECTIVE DATE: These regulations take 
effect November 27, 1998. 

FOR FURTHER INFORMATION CONTACT: 
Sharon Bobbitt, U.S. Department of 
Education, 555 New Jersey Avenue, 

NW, Room 508C, Washington, D.C. 
Telephone: (202) 219-2126. Internet: 
(Sharon — Bobbitt@ed.gov). Individuals 
who use a telecommunications device 
for the deaf (TDD) may call the Federal 
Information Relay Service (FIRS) at 1- 
800-877-8339 between 8 a.m. and 8 
p.m.. Eastern time, Monday through 
Friday. 

Individuals with disabilities may 
obtain this document in an alternate 
format (e.g., Braille, large print, 
audiotape, or computer diskette) on 
request to the contact person listed in 
the preceding paragraph. 
SUPPLEMENTARY INFORMATION: 

Background 

On March 31, 1994, President Clinton 
signed Pub. L. 103-227, which includes 
Title IX, the Educational Research, 
Development, Dissemination, and 
Improvement Act of 1994 (the Act). The 
Act restructured OERI and provided it 
with a broad mandate to conduct an 
array of research, development, 
dissemination, and improvement 
activities aimed at strengthening the 
education of all students. 

Statutory Requirements 

The Act directed the Assistant 
Secretary to develop, in consultation 




with the National Educational Research 
Policy and Priorities Board (the Board), 
such standards as may be necessary to 
govern the conduct and evaluation of all 
research, development, and 
dissemination activities carried out by 
OERI to ensure that these activities meet 
the highest standards of professional 
excellence. The Board is responsible for 
reviewing and approving the standards. 
The legislation requires that the 
standards be developed in three phases. 

In the first phase, standards were 
created and promulgated to establish the 
peer review process and evaluation 
criteria to be used for the review of 
applications for grants and cooperative 
agreements and proposals for contracts. 
The final regulations setting out these 
standards were published on September 
14, 1995 (60 FR 47808). In the second 
phase, standards were created and 
promulgated to establish the criteria to 
be used in reviewing potentially 
exemplary and promising educational 
programs. The final regulations setting 
out these standards were published on 
November 17, 1997 (62 FR 61427). 

In the third phase, which is the 
subject of these final regulations, the 
Act requires that OERI develop 
standards for evaluating and assessing 
the performance of all recipients of 
grants from and cooperative agreements 
and contracts with OERI. This 
evaluation must take place both during 
and at the conclusion of the 
performance of the grant, cooperative 
agreement, or contract, and must 
include the use of a system of peer 
review for the final assessment. 

In developing the standards, the 
Assistant Secretary was required to 
review the procedures utilized by the 
National Institutes of Health (NIH), the 
National Science Foimdation (NSF), and 
other Federal departments or agencies 
engaged in research and development 
and to solicit recommendations from 
research organizations and members of 
the general public. OERI has reviewed 
the procedures used to evaluate the 
performance of recipients of grants, 
contracts, or cooperative agreements by 
several offices within NIH and NSF, the 
Office of Energy Research in the 
Department of Energy, the Food and 
Drug Administration, the National 
Institute of Standards and Technology, 
the National Aeronautics and Space 
Administration, and the University 
Research Initiative of the Department of 
Defense. Recommendations concerning 
these standards have been obtained 
from the American Educational 
Research Association, the Council for 
Educational Development and Research, 
and the Organization of Research 
Centers. 
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Standards 

The standards have been developed 
by the Assistant Secretary in 
consultation with the Board. These 
standards cover all grants, cooperative 
agreements, and contracts administered 
by OERI, ranging from the smallest 
purchase orders and commissioned 
papers to the largest research projects 
and research centers. The standards: 

• Require at least one interim 
assessment as well as a final assessment 
of the performance of recipients of 
grants, cooperative agreements, and 
contracts. 

• Establish procedures for selecting 
peer review panels to conduct the 
assessments. 

• Establish procedures and criteria 
that the peer review panels use in 
conducting the assessments. 

• Establish specific additional criteria 
that peer review panels use in 
conducting the assessments for National 
Research and Development Centers, 
Regional Educational Laboratories, 
Field-Initiated Studies, and ERIC 
Clearinghouses. 

In an effort to fulfill the law’s 
intention of ensuring high-quality 
research, development, and evaluation, 
OERI has developed standards in which 
interim and final assessments may be 
supplemented by a self-assessment by 
the recipient of a grant, cooperative, 
agreement, or contract. The Board and 
the Assistant Secretary believe that the 
collection and review of evidence on 
one’s own performance is itself a useful 
tool for improvement. 

The Government Performance and 
Results Act requires the establishment 
of performance indicators for 
Department activities. Information 
collected pursuant to those indicators 
will be considered, as appropriate, in 
the evaluation of individual recipients. 

On February 24, 1998, the Assistant 
Secretary published a notice of 
proposed rulemaking (NPRM) for these 
standards in the Federal Register (63 FR 
9393). These final regulations contain 
four major changes from the NPRM. 
These changes are fully explained in the 
“Analysis of Comments and Changes’’ 
elsewhere in this preamble. The major 
changes pertain to clarification of the 
purpose of the regulation, how OERI 
determines the number of interim 
assessments necessary, the role of 
Department of Education staff in the 
assessments, and the use of interim 
assessments as a source of information 
for the final assessment. 

Analysis of Comments and Changes 

In response to the Secretary’s 
invitation in the NPRM, four parties 
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submitted comments on the proposed 
regulations. In addition to the public 
comment, comments from the Board's 
Subcommittee on Standards are 
addressed as required by the legislation. 
The full Board approved the final 
regulations at a meeting on September 
18, 1998. An analysis of the comments 
and of the changes in the regulations 
since publication of the NPRM follows. 

Major issues are grouped according to 
subject with appropriate sections of the 
regulations referenced in parentheses. 
Technical and other minor changes — 
and suggested changes the Secretary is 
not legally authorized to make imder the 
applicable statutory authority — are not 
addressed. 

Purpose (§702.1) 

Comments: Three comm enters 
suggested that the purpose of the 
standards be clarified. One commenter 
suggested that the standards themselves 
cannot ensure the highest standards of 
professional excellence. Another 
commenter asked specifically whether 
the purpose for conducting assessments 
was to make decisions about future 
funding or to provide a system for 
monitoring and enhancing current and 
future projects. 

Discussion: The Secretary agrees that 
the purpose of the standards should be 
clarified to go beyond their stated 
statutory purpose, which is to “ensure 
the highest standards of professional 
excellence," to include the objectives of 
continuously improving the quality of 
funded activities and of considering the 
results as one of the factors in 
determining continuation funding for 
multi-year awards. 

Changes: Section 702.1 has been 
modified to include a provision that the 
purpose of the standards is to provide 
feedback to help improve the quality of 
funded activities and to provide 
information for consideration as 
continuation funding decisions are 
made. 

Additional Activities that May be 
Evaluated (§702.3) 

Comment: One commenter thought 
that the statement that these standards 
could be applied to other activities 
funded by the Department was too 
broad and should be deleted. 

Discussion: The Secretary believes 
that this statement is necessarily broad 
to allow all Department programs to use 
these standards, when appropriate, to 
assess the performance of any of their 
funded activities without developing 
their own unique regulations. This 
statement is also consistent with the 
earlier standards which established the 
peer review process and evaluation 
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criteria to be used for the review of 
applications for grants and cooperative 
agreements and proposals for contracts. 

Changes: None. 

Number of Interim Assessments 
(§ 702.4) 

Comments: Two commenters 
suggested changes to this provision. One 
commenter suggested since there may 
be more than one interim assessment, 
that it be clear in § 702.4(d)(1). The 
OERI Board suggested that die 
requirements for a single interim 
assessment for total awards of 
$5,000,000 or less be modified to reflect 
total awards of $3,000,000 or less. 

Discussion: In response to the 
comments, the Secretary now believes 
that considerations such as difficulty in 
achieving project objectives rather than 
the dollar levels of awards should 
determine whether a particular project 
merits more than one interim 
assessment. Elimination of the dollar 
threshold clarifies the original intent of 
this section which is to require that all 
awards receive one interim assessment. 
More than one interim assessment will 
be performed only when a recipient is 
having difficulty achieving project 
objectives as determined by the initial 
interim assessment or through the 
monitoring efforts of Department of 
Education staff. The Assistant Secretary 
will make the determination of the 
number of interim assessments on a 
case-by-case basis. 

Changes: Section 702.4(b) has been 
modified to delete the dollar threshold 
and to reflect that all awards will 
receive at least one interim assessment. 
A new paragraph 702.4(c) has been 
added to clarify that the Assistant 
Secretary will require more than one 
interim assessment when a recipient has 
been identified, either in the initial 
interim review or through monitoring 
efforts of Department of Education staff, 
as having difficulty in achieving project 
objectives. Former paragraph 702.4(c) 
has been redesignated as § 702.4(d). 
Section 702.4(d)(1) has been modified to 
define an interim assessment as “any 
assessment” conducted during a 
recipient's period of performance. 

Defini tions ( § 702. 5 ) 

Comment: One commenter suggested 
that the terms referred to in this section 
include the specific definitions and not 
references to the OERI statute and to the 
Education Department General 
Administrative Regulations. 

Discussion: The Secretary believes 
that providing the citations for specific 
terms rather than the definitions 
themselves keeps regulations short and 
concise while still cross referencing 



easily accessible resources for the 
definitions. 

Changes: None. 

Characteristics of Peer Reviewers 
(§ 702.10) 

Comment: One commenter suggested 
that paragraph 702.10(a) “(4) knowledge 
of a broad range of education policies 
and practices;” be deleted from the list 
of knowledge and expertise required of 
peer reviewers, because it is redundant 
with the other criteria and is very vague. 

Discussion: The Secretary believes 
that this criterion provides for a balance 
between specific program knowledge 
and a broader perspective of education 
pohcies and practices and is therefore 
not redimdant with the other, more 
focused, characteristics required of peer 
reviewers. 

Changes: None. 

Role of Department Staff (§ 702.10) 

Comments: Two commenters 
expressed concern over the appropriate 
role of the OERI staff in the review 
process. One commenter urged the 
Department to use all outside reviewers. 
The other commenter acknowledged the 
knowledge and skills of the OERI staff 
but suggested that staff not serve as peer 
reviewers within the primary division of 
an agency in which they work and that 
each peer review panel be limited to one 
Department staff person. This 
commenter suggested that the staff focus 
on the important role of mentoring and 
designing competitions. 

Discussion: The Secretary agrees that 
the primary role of the OERI staff should 
be management of competitions 
including assessing the results of peer 
reviews and monitoring awards. The 
Secretary believes that the purpose of 
the peer review process should be to 
acquire the perspective of outside 
experts independent of OERI. The 
Secretary also believes that there may be 
exceptional circumstances where 
expertise resides in OERI or in the 
Department, or where outside reviewers 
are not required such as in the review 
of small purchase orders. The 
exceptions should be determined by the 
Assistant Secretary. 

Changes: Section 702.10(d) has been 
reworded to preclude OERI and other 
Department staff from serving as peer 
reviewers except in exceptional 
circumstances as determined by the 
Assistant Secretary. 

Conflict of In terest ( § 702. 11) 

Comment: One commenter was 
concerned that while the conflict of 
interest requirements were “legally 
correct” they failed to address the 
problem occasioned by reviewers who 
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may have ideological or methodological 
view points that differ from those of the 
recipient to be evaluated, or who are 
affiliated with competing institutional 
organizations. 

Discussion: The commenter appears 
to be concerned that the proposed 
conflict of interest provision does not 
address the potential problem of bias on 
the part of a panel against a particular 
grantee on ideological or other grounds. 
The Secretary first believes that it is 
essential to retain the present language, 
which parallels the provision in the 
standards at 34 CFR 701.11(c), because 
it highlights the important issue of 
improper financial gain or the 
appearance of improper gain. However, 
the Secretary agrees that adding a 
requirement to the effect that panels 
selected by the Assistant Secretary 
reflect a broad range of perspectives 
could strengthen the regulation. 

Changes: A new paragraph “(c)” has 
been added to § 702.13 requiring the 
Assistant Secretary, to the greatest 
extent feasible, to select peer reviewers 
for each evaluation who represent a 
broad range of perspectives. 

Sources of Information (§§ 702. 22 an d 
702.23) 

Comment: One commenter suggested 
that the use of Government Performance 
and Results Act (GPRA) information 
should be encouraged rather than 
required for both interim and final 
assessments. The commenter is 
concerned that information currently 
being collected under GPRA to evaluate 
the effectiveness of a program or a 
system-level activity will not provide 
information relevant to the assessment 
of individual awards under that 
program or system-level activity and 
therefore should not be required. 

Discussion: The Secretary agrees that 
information obtained by GPRA-related 
reports on the effectiveness of a program 
or system level activity, e.g., how 
effectively a program is meeting the 
overall objectives defined for it in its 
authorizing legislation, may not 
necessarily include information related 
to an individual award being reviewed 
imder this regulation. However, the 
Secretary believes that information on 
the effectiveness of the particular 
program under which a recipient 
receives funding will help to provide a 
context for the review of an individual 
award and must be considered by the 
panel. Moreover, these regulations make 
it clear that the GPRA information is 
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only one of a number of sources used in 
conducting the review. 

Changes: None. 

Comment: One commenter suggested 
that the findings and information from 
interim assessments would be an 
important source of information for the 
final assessments and should be 
included imder § 702.23(a). 

Discussion: The Secretary agrees that 
the results of interim assessments 
should be a source of information for 
final assessments. 

Change: Section 702.23(a) has been 
modified to add a new paragraph 
(§ 702.23(a)(5)) to require that the results 
of interim assessments be considered as 
a source of information for final 
assessments. 

Evaluation Criteria (§702.24) 

Comments: Two commenters 
suggested changes to this section. One 
commenter suggested that there be a 
single menu of criteria for the standards, 
because the proposed menu is too long. 
The second commenter suggested that 
since Field Initiated Studies are not 
likely to provide services, the word 
“services” be deleted from the criterion 
in §702.24(e)(4)(ii): “* * * addresses 
issues of national significance through 
its products or services, or both.” 

Discussion: The Secretary believes the 
cmrent menu approach provides a 
comprehensive strategy for assessing the 
performance of all activities, ranging 
from the smallest purchase order to the 
largest research investments. The 
categories in the regulation reflect the 
specific authorities in the OERI statute. 
In addition, the menu provides for other 
criteria for future research investments 
that do not fit within the statutory 
authorities yet also must be assessed. A 
single menu would, of necessity, be too 
generic to apply to the wide range of 
activities covered by these standards. 
The Secretary agrees that assessing 
“services” is not appropriate for Field 
Initiated Studies projects. 

Change: Section 702.24(e)(4)(ii) has 
been modified to delete the word, 
“services.” 

Paperwork Reduction Act of 1995 

Under the Paperwork Reduction Act 
of 1995, no persons are required to 
respond to a collection of information 
unless it displays a valid 0MB control 
number. The valid 0MB control number 
assigned to the collection of information 
in these final regulations is displayed at 
the end of the affected sections of the 
regulations. 



Assessment of Educational Impact 

In the NPRM the Secretary requested 
comments on whether the proposed 
regulations would require transmission 
of information that is being gathered by 
or is available from any other agency or 
authority of the United States. 

Based on the response to the NPRM 
and on its own review, the Department 
has determined that the regulations in 
this document do not require 
transmission of information that is being 
gathered by or is available from any 
other agency or authority of the United 
States. 

Electronic Access to This Document 

Anyone may view this document, as 
well as all other Department of 
Education documents published in the 
Federal Register, in text or portable 
document format (pdf) on the World 
Wide Web at either of the following 
sites: 

http://ocfo.ed.gov/fedreg.htm 
http ://Avww.ed.gov/ news.html 

To use the pdf you must have the 
Adobe Acrobat Reader Program with 
Search, which is available free at either 
of the previous sites. If you have 
questions about using the pdf, call the 
U.S. Government Printing Office at (202) 
512-1530 or, toll free, at 1-888-293- 
6498. 

Anyone may also view these 
documents in text copy only on an 
electronic bulletin board of the 
Department. Telephone: (202) 219-1511 
or, toll free, 1-800-222-4922. The 
documents are located imder Option 
G — Files/ Announcements, Bulletins and 
Press Releases. 

Note: The official version of this document 
is the document published in the Federal 
Register. 

List of Subjects in 34 CFR Part 702 

Education, Educational research. 
Reporting and recordkeeping 
requirements. 

(Catalog of Federal Domestic Assistance 
Number does not apply.) 

Dated: October 22, 1998. 

C. Kent McGuire, 

Assistant Secretary for Educational Research 
and Improvement. 

The Secretary amends Chapter VII of 
Title 34 of the Code of Federal 
Regulations by adding a new Part 702 to 
read as follows: 
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PART 702— STANDARDS FOR 
CONDUCT AND EVALUATION OF 
ACTIVITIES CARRIED OUT BY THE 
OFFICE OF EDUCATIONAL 
RESEARCH AND IMPROVEMENT 
(OERI)— EVALUATION OF THE 
PERFORMANCE OF RECIPIENTS OF 
GRANTS, COOPERATIVE 
AGREEMENTS, AND CONTRACTS 

Subpart A— General 

Sec. 

702.1 What is the purpose of these 
standards? 

702.2 What activities must be evaluated by 
these standards? 

702.3 What additional activities may be 
evaluated by these standards? 

702.4 When is performance assessed under 
these standards? 

702.5 What dehnitions apply? 

Subpart B — Selection of Peer Review 
Panels 

702.10 What are the characteristics of peer 
reviewers? 

702.11 What constitutes a conflict of 
interest for grants and cooperative 
agreements? 

702.12 What constitutes a conflict of 
interest for contracts? 

702.13 How are peer reviewers selected for 
panels? 

Subpart C— The Evaluation Process 

702.21 How does a peer review panel 
evaluate the performance of a recipient? 

702.22 What information does a peer 
review panel consider for an interim 
assessment? 

702.23 What information does a peer 
review panel consider for a final 
assessment? 

702.24 What evaluation criteria must be 
used for performance assessments? 

Authority: 20 U.S.C. 6011(i), unless 
otherwise noted. 

Subpart A— General 

§ 702.1 What Is the purpose of these 
standards? 

(a) The standards in this part 
implement section 912(i) of the 
Educational Research, Development, 
Dissemination, and Improvement Act of 
1994 (the Act). 

(b) These standards establish criteria 
and a peer review process to provide 
recipients of OERI grants, cooperative 
agreements and contract awards with 
assessments of their projects. 

(1) The purpose otthe assessments is 
to provide feedback to recipients to 
improve the quality of funded activities 
and to provide information to OERI as 
it determines if a recipient of a multi- 
year award merits continuation funding. 

(2) The criteria and peer review 
process are intended to address the 
statutory requirement that the research, 
development, and dissemination 
activities carried out by the recipients of 







grants from and contracts and 
cooperative agreements with the Office 
of Educational Research and 
Improvement (OERI) meet the highest 
standards of professional excellence. 

(Authority: 20 U.S.C. 6011(i)(2)(F)) 

§ 702.2 What activities must be evaluated 
by these standards? 

These standards apply to activities 
carried out by OERI using funds 
appropriated under section 912(m) of 
the Act including activities carried out 
by the following entities or programs: 

(a) The National Education Research 
Institutes. 

(b) The Office of Reform Assistance 
and Dissemination. 

(c) The Educational Resources 
Information Center. 

(d) The Regional Educational 
Laboratories. 

(e) The Teacher Research 
Dissemination Demonstration Program. 

(f) The Ck)als 2000 Community 
Partnerships Program. 

(g) The National Educational Research 
Policy and Priorities Board. 

(Authority: 20 U.S.C. 6011(i)(l)) 

§ 702.3 What additional activities may be 
evaluated by these standards? 

The Secretary may apply these 
standards to other activities funded by 
the Department, as appropriate. 

(Authority: 20 U.S. C 6011 (i)(l)) 

§ 702.4 When 1s performance assessed 
under these standards? 

(a) The Secretary will assess the 
performance of recipients of OERI 
grants, contracts, and cooperative 
agreements subject to these standards 
during and at the conclusion of their 
period of performance. 

(b) The Department requires at least 
one interim assessment by a peer review 
panel for all awards. 

(c) The Assistant Secretary will 
approve and require more than one 
interim assessment when an award is 
identified, either by the initial interim 
review or by Department of Education 
staff monitoring the award, as having 
difficulty in achieving project 
objectives. 

(d) A final assessment by a peer 
review panel is required for all awards. 

(e) As used in this part — 

(1) Interim assessment is any 
assessment conducted during a 
recipient's period of performance. 

(2) Final assessment is one conducted 
at the conclusion of a recipient’s period 
of performance. 

(Authority: 20 U.S.C. 6011(i)(2)(F)) 

§ 702.5 What definitions apply? 

(a) Definitions in the Educational 
Research, Development, Dissemination, 



and Improvement Act of 1994. The 

following terms used in this part are 

defined in 20 U.S.C. 6011(l)(l): 

Development 

Dissemination 

Educational Research 

(b) Definitions in the Education 
Departmen t Gen eral A dministrati ve 
Regulations. The following terms used 
in this part are defined in 34 CFR 77.1: 
Application 

Award 

Department 

Grant 

Project 

Secretary 

(c) Definitions in the Federal 
Acquisition Regulation. The following 
term used in this peirt is defined in 48 
CFR Chapter 1: Contract Proposal. 
(Authority: 20 U.S.C. 601l(i)(2)(F) 

Subpart B — Selection of Peer Review 
Panels 

§ 702.10 What are the characteristics of 
peer reviewers? 

(a) The Assistant Secretary selects 
each peer reviewer. Each peer reviewer 
must have the necessary Imowledge and 
expertise in the area of the project being 
reviewed to evaluate the performance of 
a recipient. This experience may 
include — 

(1) Expert knowledge of subject matter 
in the area of the activities to be 
reviewed; 

(2) Expert knowledge of theory or 
methods or both in the area of the 
activities to be reviewed; 

(3) Practical experience in the area of 
the activities or type of institution or 
both to be reviewed; 

(4) Knowledge of a broad range of 
education policies and practices; 

(5) Experience in managing complex 
organizations; or 

(6) Expertise and experience in 
evaluation theory and practice. 

(b) Each peer reviewer must be free of 
conflict of interest, as determined in 
accordance with § 702.11 or § 702.12. 

(c) The Assistant Secretary may solicit 
nominations for peer reviewers from 
professional associations, nationally 
recognized experts, and other sources. 

(d) OERI and other Department staff 
who possess the qualifications in 
paragraphs (a) and (b) of this section 
may serve as peer reviewers only in 
exceptional circumstances as 
determined by the Assistant Secretary. 

(Authority: 20 U.S.C. 6011(i)(2)(B)) 

§ 702. 11 What constitutes a conflict of 
Interest for grants and cooperative 
agreements? 

A peer reviewer assessing the 
performance of the recipient of a grant 



37 



57574 



Federal Register / Vol. 63, No. 207 /Tuesday, October 27, 1998 /Rules and Regulations 



from or cooperative agreement with 
OERI is considered an employee of the 
Department for the purposes of conflict 
of interest analysis. As an employee of 
the Department, the peer reviewer is 
subject to the provisions of 18 U.S.C. 

208, 5 CFR 2635.502, and the 
Department's policies used to 
implement those provisions. 

(Authority: 20 U.S.C. 6011{i)(2)(B)) 

§702.12 What constitutes a conflict of 
Interest for contracts? 

A peer reviewer assessing the 
performance of the recipient of a 
contract with OERI is considered an 
employee of the Department in 
accordance with the Federal Acquisition 
Regulation (FAR), 48 CFR 3.104-4(h)(2). 
As an employee of the Department, the 
peer reviewer is subject to the 
provisions of the FAR, 48 CFR Part 3, 
Improper Business Practices and 
Personal Conflict of Interest. 

(Authority: 41 U.S.C. 423) 

§ 702.1 3 How are peer reviewers selected 
for panels? 

(a) The Assistant Secretary assigns 
peer reviewers to panels that conduct 
the performance assessments. 

(bj The Assistant Secretary may 
establish panels by category of recipient, 
such as a panel to review the 
performance of all Regional Educational 
Laboratories. Each recipient is evaluated 
individually by reviewers who have 
been assigned to this type of panel. 

(c) In establishing panels, tne 
Assistant Secretary, to the greatest 
extent feasible, selects peer reviewers 
for each evaluation who represent a 
broad range of perspectives. 

(Authority: 20 U.S.C. 6011(i)(2)(B)) 



Subpart C — The Evaluation Process 

§ 702.21 How does a peer review panel 
evaluate the performance of a recipient? 

(a) In each evaluation, a peer review 
panel — 

(1) Considers relevant information 
about the recipient’s performance, as 
described in §§ 702.22 and 702.23; and 

(2) Makes judgments about the 
recipient’s performance, using the 
criteria in § 702.24. 

(b) Each peer reviewer prepares a 
report based on the reviewer’s 
assessment of the quality of the project 
according to the evaluation criteria. 

(c) After each peer reviewer has 
evaluated each project independently, 
the panel may be convened to discuss 
the strengths and weaknesses of the 
project. Each reviewer may then 
independently re-evaluate each project 
with appropriate changes made to the 
written report. 

O 



(d) The report of the interim 
assessment must include any 
recommendations the peer reviewer 
may have for improving the recipient’s 
performance. 

(e) The report of the final assessment 
must contain each peer reviewer’s 
evaluative summary of the recipient’s 
performance, from the beginning of the 
contract, grant, or cooperative 
agreement to its conclusion. 

(Authority: 20 U.S.C. 6011(i)(2)(F)) 

§702.22 What Information does a peer 
review panel consider for an Interim 
assessment? 

(а) Sources of information for the 
interim assessment must include — 

(1) The original request for proposals 
or grant announcement and the contract 
proposal or grant application; 

(2) Documentation of any changes in 
the work described in the contract, 
grant, or cooperative agreement, 
including reasons for the changes; 

(3) Any progress reports delivered to 
the Department or made available to the 
public by the recipient; 

(4) Examples of products delivered to 
the Department or made available to the 
public by the recipient; 

(5) Any relevant reports written by 
OERI staff, including reports of site 
visits by OERI staff; 

(б) Any performance evaluations 
conducted under the FAR or the 
Education Department General 
Administrative Regulations (34 CFR Part 
75). 

(7) Any relevant information provided 
by the recipient in response to 
Ciovemment Performance and Results 
Act (GPRA) (Pub. L. 103-62) 
requirements; and 

(8) Any reports from program 
evaluations commissioned by the 
Department. 

(d) Sources of information for the 
interim assessment may also include — 

(1) A self-assessment, prepared by the 
recipient, addressing the criteria in 
§702.24; 

(2) One or more site visits by the peer 
review panel; 

(3) One or more oral or written 
presentations to the panel by the 
recipient describing its performance; or 

(4j Other information about the 
recipient’s performance. 

(Approved by the Office of Management and 
Budget under control number 1850-0746) 
(Authority: 20 U.S.C. 6011(i)(2)(F)) 

§702.23 What Information does a peer 
review panel consider for a final 
assessment? 

(a) Sources of information for the final 
assessment must include — 

[ 1 ) The original request for proposals 
or application notice and the contrac^ ^ 



proposal or grant application, together 
with documentation of any changes in 
the work described in the proposal or 
application, including reasons for the 
changes; 

(2) If consistent with the recipient’s 
contract, grant, or cooperative 
agreement with OERI, a written report 
or oral presentation or both by the 
recipient summarizing its activities and 
accomplishments; 

(3) Any relevant information provided 
by the recipient in response to 
Government Performance and Results 
Act (GPRA) (Pub. L. 103-62) 
requirements; 

(4) Any reports from program 
evaluations commissioned by the 
Department; and, 

(5) Any relevant information provided 
by the interim assessment. 

(b) The final assessment may also 
include other sources of information, 
such as one or more of those listed in 
§702.22. 

(Approved by the Office of Management and 
Budget under control number 1850-0746) 
(Authority: 20 U.S.C. 6011(i)(2)(F)) 

§ 702.24 What evaluation criteria must be 
used for performance assessments? 

(a) Peer reviewers (and those 
recipients who conduct self-evaluations) 
shall use the criteria in paragraph (b) of 
this section to assess performance and, 
in case of interim assessments, to 
identify areas in which the performance 
of recipients may need improvement. 

(b) ’me following evaluation criteria 
are to guide the assessment process 
undertaken by peer reviewers. The peer 
reviewers determine the extent to which 
recipients meet these criteria: 

(Ij Implementation and management. 
(i) Peer reviewers shall consider the 
degree to which the recipient has fully 
executed its program of work. In doing 
so, peer reviewers shall consider 
evidence on the extent to which the 
recipient completes the work described 
in the approved application or contract, 
including any approved modifications, 
in the time period proposed and in an 
efficient manner. 

(ii) In examining the degree of 
implementation, peer reviewers may 
also consider evidence on the extent to 
which — 

(A) The recipient implements and 
utilizes a quality assurance system for 
its products or services or both; and 

(B) The recipient conducts self- 
assessment or self-evaluation activities, 
including periodically seeking out 
independent critiques and evaluations 
of its work, and uses the results to 
improve performance. 

(2) Quality, (i) Peer reviewers shall 
consider the degree to which the 
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recipient’s work approaches or attains 
professional excellence. In determining 
quality, peer reviewers shall consider 
evidence on the extent to which — 

(A) The recipient utilizes processes, 
methods, and techniques appropriate to 
achieve the goals and objectives for the 
program of work in the approved 
application; and 

(B) The recipient applies appropriate 
processes, methods, and techniques in a 
manner consistent with the highest 
standards of the profession. 

(ii) In determining quality, peer 
reviewers may also consider the extent 
to which the recipient conducts a 
coherent, sustained program of work 
informed by relevant research. 

(3) Utility, (i) In determining the 
utility of the recipient’s products or 
services or both, peer reviewers shall 
consider evidence on the extent to 
which the recipient’s work (including 
information, materials, processes, 
techniques, or activities) is effectively 
used by and is useful to its customers 
in appropriate settings. 

(ii) In determining utility, peer 
reviewers may also consider the extent 
to which the recipient has received 
national recognition; e.g., articles in 
refereed journals and presentations at 
professional conferences. 

(4) Outcomes and impact, (i) Peer 
reviewers shall consider the results of 
the recipient’s work. In examining 
outcomes and impact, peer reviewers 
shall consider evidence on the extent to 
which — 

(A) The recipient meets the needs of 
its customers; and 

(B) The recipient’s work contributes 
to the increased knowledge or 
understanding of educational problems, 
issues, or effective strategies. 

(ii) In examining outcomes and 
impact, peer reviewers may also 
consider the extent to which recipients 
address issues of national significance 
through its products or services or both. 

(c) For National Research and 
Development Centers, peer reviewers 
also shall consider evidence on the 
extent to which recipients meet the 
following criteria: 

(1) Quality, (i) The recipient uses a 
well-conceptualized framework and 



sound theoretical and methodological 
tools in conducting professionally 
rigorous studies; and 

(ii) The recipient conducts work of 
sufficient size, scope, and duration to 
produce sound guidance for 
improvement efforts and future 
research. 

(2) Utility. The recipient documents, 
reports, and disseminates its work in 
ways to facilitate the effective use of its 
work in appropriately targeted settings. 

(3) Outcomes and impact, (i) The 
recipient’s work contributes to the 
development and advancement of 
theory in the field of study, including its 
priority area; and 

(ii) The recipient addresses issues of 
national significance through its 
products or services or both. 

(d) For the Regional Educational 
Laboratories, peer reviewers also shall 
consider evidence on the extent to 
which recipients meet the following 
criteria: 

(1) Quality, (i) The recipient utilizes a 
well-conceptualized framework and 
sound theoretical and methodological 
tools in conducting professionally 
rigorous studies; 

(ii) The recipient conducts work of 
sufficient size, scope, and duration to 
produce sound guidance for 
improvement efforts; and 

(lii) The recipient’s products are well 
tested and based on sound research. 

(2) Utility. The recipient documents, 
reports, and disseminates its work in 
ways to facilitate its effective use in 
appropriately targeted settings, 
particularly in school improvement 
efforts of States and localities. 

(3) Outcomes and impact, (i) The 
recipient assists States and localities to 
implement comprehensive school 
improvement strategies through the 
provision of research-based information 
(including well-tested models and 
strategies), materials and assistance; and 

(ii) The recipient’s work results in 
widespread access to information 
regarding research and best practices, 
particularly within its region. 

(e) For Field-Initiated Studies, peer 
reviewers also shall consider evidence 
on the extent to which recipients meet 
the following criteria: 



(1) Implementation and management. 
The recipient’s work responds to the 
goals, objectives and mission of the 
National Institute from which it is 
funded. 

(2) Quality. The recipient utilizes a 
well-conceptualized framework and 
sound theoretical and methodological 
tools in conducting professionally 
rigorous studies. 

(3) Utility. The recipient documents, 
reports, and disseminates its work in 
ways to facilitate its effective use in 
appropriately targeted settings. 

(4) Outcomes and impact, (i) The 
recipient’s work contributes to the 
development and advancement of 
theory and knowledge in the field of 
study; and 

(ii) The recipient addresses issues of 
national significance through its 
products. 

(f) For the ERIC Clearinghouses, peer 
reviewers also shall consider evidence 
on the extent to which recipients meet 
the following criteria: 

(1) Quality. The recipient applies an 
integrated approach to acquiring and 
disseminating significant and high- 
quality educational literature and 
materials to maintain and enhance the 
ERIC database. 

(2) Utility. The recipient contributes 
to the development of the ERIC database 
as a source of literature and materials 
that reflects trends and issues within its 
scope. 

(3) Outcomes and impact, (i) The 
recipient meets the informational and 
educational needs of its customers 
through dissemination and outreach 
approaches and the development of an 
array of print and non-print materials; 
and 

(ii) The recipient provides national 
leadership on the use of current 
computer, networking, and information 
technology. 

(Approved by the Office of Management and 
Budget under control number 1850-0746) 
(Authority: 20 U.S.C. 6011(i)(2)(F)) 
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